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(57) On a BD-ROM, there is recorded is a digital 
stream into which video and audio streams are multi- 
plexed. The video stream Is composed of a plurality of 
pictures that together represent video. The graphics 
stream includes a PES packet storing state control in- 
formation (ICS) and PES packets containing graphics 
data (ODSs). The graphics data constructs an interac- 
tive display. The state corttrbl Information defines the 



control so as to change the states of buttons presented 
oh the interactive display according to the reproduction 
proceeding and user operations. 

The ICS Is attached with a PTS showing the display 
timing of a picture to be synchronized with the Interactive 
display. The ODS is attached with a PTS shoeing the 
timing for decoding the graphics data. The timing shown 
in the ODS is prior to tne display timing. 1 
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Description 

TECHNICAL FIELD 

9 [0001] The present Invention relates to a recording medium such as a BD-ROM for distribution of movies, and also 
to a reproduction apparatus for such a recording medium. More particularly, the present invention relates to an im- 
provement in interactive control techniques. 

BACKGROUND ART 

to 

[0002] It has long been desired to realize such Interactive control that buttons appear on a display screen as e video 
stream is reproduced and that the reproduction proceeds in accordance with user operations made to the buttons. A 
DVD is a breakthrough recording medium realizing such reproduction control. The synchronous presentation of buttons 
with a video stream Is established through the use of time stamps set to cause the buttons to appear at specific points 
on the reproduction time axis of the video stream. 

[0003] In order to realize the interactive control, however, it is not sufficient to record onto a recording medium graph- 
ics data used for rendering the buttons. A reproduction apparatus needs to be controlled so as to change the state of 
each button displayed on a screen, according to a user operation or to the proceeding of video stream reproduction; 
To realize the Interactive control, a stream into which audio and video streams are multiplexed (Video Objects) is 

20 recorded on a DVD, and a NAVI pack containing state control Information is provided at the beginning of each VOBU. 
The VOBU includes one GOP of the video stream, and also includes audio data and graphics data, which Is supple- 
mental to the video stream, both to be read simultaneously with the GOP from the DVD. The state control Information 
is used to change, according to. a user operation, the state of each button displayed on the screen. The NAVI pack 
contains information defining a transfer rate and a buffer size that each stream requires in handling the GOP. Since 

25 the DVD stores the state control information In NAVI packs, the button states can be changed with the time accuracy 
of GOPs. FIG. 1 1llustrates the Interactive control described above. In the figure, the lowest level shows data allocation 
. on the DVD. It is shown that state control Information Is contained In a NAVI pack. The state control Information remains 
valid during a time period of the GOP to which the NAVI pack belongs. Each graphics object Is contained in a PES 
packet, and displayed at the same timing with a picture to be synchronized with the graphics object. This prior art 

30 technique Is disclosed for example in JP patent No. 2813245. 

[0004] It should be noted here that In DVD authoring, the structures of GOPs and VOBUs are not determined until 
elementary streams of video, audio, and graphics data are encoded and ready to be multiplexed into one VQB. The 
multiplexing Is a final stage of authoring. That Is, the state control Information cannot be Incorporated Into a VOQ before 
this final stage, and thus testing of how the buttons change on a display screen can not be carried out any! earlier. 

35 Consequently, il Is often the case that bugs are found right before the shipment and the developers are forced to make 
corrections in a rush. In addition, without enough time for the testing, It is risky to Incorporate buttons of complex 
animation into a movie. For the reasons stated above, It is common In current authoring to Incorporate relatively simple 
buttons, such as one which changes in color in response to a user operation. 

40 DISCLOSURE OF THE INVENTION 

[0006] The present invention alms to provide a recording medium which allows the button state changes to be tested 
at an early stage of authoring. 

[0006] It has been an effective scheme to store state control Information In a NAVI pack of each VOBU when con- 
49 siderlng a DVD's data read rate. This Is because the state control Information Is read from the DVD simply by reading 
the NAVI pack, so that the bandwidth required for reading is low. 

[0007] However, the scheme la not as effective when considering a rate at which a BD-ROM Is read. The BDrROty's 

read rate is much higher than that of DVD* s, so that it is no longer important to restrict the bandwidth. 

[0008] In view of the advances In recording mediums end to achieve the above aim, a recording medium according 

so to the present invention has recorded thereon a digital stream :o which a video stream and a graphics stream are 
multiplexed. The graphics stream Includes graphics data and s;* : control Information integrated therein. The graphics 
data Is used to compose an interactive display. The state cor;- information Is used to cause the interactive 'display 
to change to a different state In response to a reproduction proceeding of the video stream and a user operation. 
According to the recording medium of the present invention, the Information for causing the button state chpnge Is 

55 integrated with the graphics data Into the graphics stream. Thus, as soon as the graphics stream is generajed, the 
validation test can be carried out to check how the button state changes in accordance with the reproduction propeeding. 
It is no longer necessary to wait for the completion of video stream encoding or of stream multiplexing. Since the 
verification test of button state changes can be carried out at an early stage of the authoring (prior to the strejammul- 
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tiplexing), the undesirable jpossibility is reduced that an error is found in the recording medium right bef ore the shipment 
and the developers are forced to rush. Also, since the graphics stream can be subjected to the verif ication test inde- 
pendently of the other streams, there is provided a better environment for incorporating buttons of complex animation 
into r a movie. 

s . [0009] Here, the state control information in each of the plurality of display sets may Include an update flag. When 
set to ON, the update flag indicates that the display set Is identical lo an immedately preceding display set with respect 
to the state control Information and the graphics data, except for a button command. When set to OFF, the update flag 
indicates that the display set is Identical to an immediately preceding, display set with respect to the state control 
information and the graphics data. The button command may be for execution by a reproduction apparatus upon 

10 activation of an associatedbutton on the Interactive display. . 

[0010] With the above state structure, such a title can be produced that the reproduction branches to one out of a 
plurality of reproduction paths In accordance with button commands. Forexample, atitle is a quiz game and two oteplay 
sets present a question for a user to answer. The title may be constructed to branch to a more and more disadvanta- 
geous reproduction path for the user as the user response delays. 

is [001 1] Here, each of the n buttons may have a number assigned thereto. Each of the n pieces of button information 
may include a number assigned to an associated button and a f jag showing whetherthe button is numerically selectable 
with the number. With this structure, discrete numbers can be assigned to the buttons, which is convenient for producing 
such a tide as a list of baseball players. 

[0012] To be more specific, such button commands are prepared that the reproduction branches to scenes showing 
so play of the baseball players. In addition/the player numbers are assigned to the button commands . With this arrange- 
ment, the reproduction path to trie scenes of a specific baseball player is taken at a numeric Input of the player number. 

BRIEF DESCRIPTION OF DRAWINGS . 

25 [0013] 

FIG. 1 Is a view Illustrating interactive control on a DVD; 

FIG. 2A is a view showing a usage pattern of a recording medium according to the present Invention; 
FIG. 2B is a view showing keys of a remote controller 400 for receiving user operations to an interactive display; 
30 FIG. 3 Is a view showing a structure of a BD-ROM; 

FIG. 4 is a view schematically shows a structure of an AV Clip; 
FIG. 5 Is a view showing an Internal structure of Clip information; 
FIG, 6 is a view showing an internal structure of PlayList Information; 
I FIG. 7 fs a view schematically illustrating indirect references by the PlayList information; 

35 FIG. 8A is a view showing a structure of a graphical stream; 

F*G. 8B is a view showing internal structures of an ICS and an ODS; 
FIG. 9 is a view showing a logicai structure defined by various types of functional segments; 
FIG. 10A is a view showing a data structure of an ODS defining a graphics object; 
FIG. 1 0B is a view showing a data structure of a PDS; 
<to FIG. 1 1 is a view Showing a data structure of an Interactive Composition Segment; 

' FIG. 12 is a view showing the relation between ODSs and ICS contained within a DS (n); 

FIG. 13 is a view showing a composite screen at the display timing of an arbitrary picture pt1 ; 
FIG. 14 is a view showing an example of button information setting within an ICS; 
. FIG. 15 is a view illustrating button state transitions of buttons A-D; 
45 FIG; 1 6 is a view showing examples of graphical reorientations defined by ODSs 11 , 21 , 31 , and 41 ; 

FIG. 1 7 is a view showing examples of graphical representations defined by ODSs 11 -1 9 associated with the button 
A; t : • • /■ .. 

. FIG. 1 8 is a view showing an example of an ICS and ODSs included In a DS; 
. FIG. 19 is a view showing the order of ODSs and button-state groups within a DS; 
so FIG. 20 a view illustrating button state transitions on an Interactive display defined by the button-state groups 

shown in FIG. 19; 

FIG. 21 is a view showing the order of ODSs within a DS; 

FIG; 22 isBvlew Illustrating the timing of synchronous display defined by an ICS; ■ , 

FIG. 23 is a view Illustrating the DTS and PTS setting in the case where a plurality of ODSs constitutes an Initial . , 
55 interactive display and a defaurLselected.button.number is valid; 
: FIG. 24 is a view illustrating the setting of DTS and PTS in the case where an initial interactive display is constituted 

by a plurality of ODSs and a default selected button number is invalid; 
FIG. 25 is a view showing an internal structure of a reproduction apparatus; 
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FIG. 26 a timing chart of the pipeline processing performed by the reproduction apparatus; 

FIG. 27 a timing chart of the pipeline processing performed by the reproduction apparatus in the case where a 

default selected button is not specified; 

FIG. 28 Is a flowchart of the operations of a controller 20 for executing a LlnkPL function; 
5 FIG. 29 Is a flowchart of the load processing of functional segments; 

FIG. 30 Is a view illustrating how the load processing is performed at the time of a skfc operation; 

FIG. 31 Is a view illustrating how a DS 1 0 is loaded to a coded data buffer 1 3 of the reproduction apparatus; 

FIG. 32 is a view illustrating normal reproduction; 

FIG. 33 is a view Illustrating how DSs 1 , 1 0, 20 are loaded when the norma) reproduction is performed as Illustrated 
to in FIG. 32; 

FIG. 34 is a flowchart of the main routine of processing performed by the graphics controller 17; 
FIG. 35 is a flowchart of the synchronization control based on time stamps; 
FIG. 36 is a flowchart of the rendering to the graphics plane 8; 

FIG. 37 Is a flowchart of the processing for automatic activation of a default selected button; 
f5 FIG. 38 is a flowchart of the processing for animated presentation; 

FIG. 39 is a flowchart of the UO processing; 
FIG. 40 is a flowchart of the current button change processing; 
FIG. 41 is a flowchart of the numeric input processing; 

FIG. 42 Is a view showing an internal structure of the reproduction apparatus capable of producing a click sound; 
20 FIG. 43 is a view showing a data structure of an ICS for reproducing a click sound; 

FIG. 44A is a view showing the state control Information including button Jnfo (1 ) and buttonjnfo (2); 

FIG. 44B Is a view showing the reading process of the ICS including the state control information; 

FIG. 44C is a view showing an example of three buttons (buttons A, B, and C) that are diagonally aligned on an 

interactive display and an example of button state Information associated with the buttons; 
25 FIGs. 45 A and 45B are views illustrating the reproduction control of the click sound data according to the ICs read 

to the preload memory 21 ; 

FIG. 45C is a view showing an example of three buttons (buttons A, B, and C) that are laterally aligned on an 
interactive display and an example of button state Information associated with the buttons 
FIG. 46 to -a flowchart of the manufacturing process of a BD-ROM according to a third embodiment of the present 
30 Invention; 

FIG. 47 is a view showing an ICS according to one modification of the present Invention; and 
FIG. 4B Is a view showing an ICS defining a click sound for each key of the remote controller. 

BEST MODE FOR CARRYING OUT THE INVENTION 

35 

(FIRST EMBODIMENT) 

[0014] The following Is a description of a recording medium according to a first embodiment of the present invention. 
. First, a description Is given to usage of the recording medium, which Is one form of practicing the present invention. 

40 FIG. 2A shows a usage pattern of the recording medium according to the present invention. In FIGs. 2, a 3D- ROM 
100 is a recording medium according to the present invention. The BD-ROM 100 is used for providing movies to a 
home theater system composed of a reproduction apparatus 200, a television 300, and a remote controller ^00. The 
remote controller 400 receives user operations Instructing to change the state of Interactive display, and iq closely 
related to the recording medium according to the present invention. FIG. 23 shows keys of the remote controller 400; 

45 As shown In the figure, the remote controller 400 has a Move Up key, a Move Down key, a Move Right key, ancj a Move 
Left key. Each button displayed on the Interactive display has three states: the normal state; the selected state; and 
the activated state. The Move Up, Move Down, Move Right, and Move Left keys are used to receive user operations 
for causing the button state to change In the order of the normal state -* the selected state -> the activated state, for 
example. When a button has the normal state, the button Is simply displayed. When a button has the selected stated, 

50 the button is currently focused as a result of a user operation but not yet activated. When a button has the activated 
state, the button has been activated. When a specific button has the selected state on an Interactive display, tjie Move 
Up key Is used to put a button displayed above the currently selected button to the selected state. The Move Down 
key Is used to put a button displayed below the currently selected button to the selected state. The Move Right key Is 
used to put a button displayed right to the currently selected button to the selected state. The Move Left key is used 

55 to out a button displayed left to the currently selected button to the selected state. ! 

[001 5] An Activate key Is used to put a button having the selected state to the activated state (to activate the purrentiy 
selected button) . Numeral keys "0" to n 9 n are used for numeric selection so as to put a button having assigned an 
Inputted value to the selected state. A ''+10° key is used to add the value of 10 to a value having been entered. Note 
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that tho "0* key and the a +10" key are both used to enter a two-digit value. Thus, one of the "b" and "+10" keys may 
be sufficient instead of both. 

[0016] This concludes the description of the usage of recording medium according to the present invention. 
[0017] Next, a description is given to production of the recording medium, which is another form of practicing the 

5 present Invention. The recording medium can be embodied by making Improvements to an application layer of a 
BD-ROM. FIG. 3 shows an example structure of the BD-ROM 100. 

[0018] In the figure, the BD-ROM 100 is shown on the fourth level, and a BD-ROM* s track is shown on the third 
j level. In the figure, the track Is stretched out Into a straight line, although the track in practice spirals outwards from 
• the center of the BD-ROM. The track Is composed of a lead-in area, a volume area, and a lead-out area. The volume 
to area has a layer model of a physical layer, a file system layer, and an application layer. The first level shows, in a 
directory structure, a format of the application layer (application format) of the BD-ROM. As illustrated, the BI>ROM . 
has a BDMV directory below a ROOT directory. The BbMV directory contains files, such as XXX.M2TS, XXX.CLPI, 
and YYY.MPLS. The BD-ROM of the present Invention can be produced by creating an application format as shown 
In the figure. 

15 [0019] Now, a description Is given to each of the files used in the application format. First, an AV Clip (XXX.M2T S 
fiie) is described. 

[0020] The AV Clip (XXX.N2TS) is a digital stream compliant with an MPE6-TS (transport stream) format, and ob- 
tained by multiplexing a video stream, one or more audio streams, a presentation graphics stream, and an interactive 
graphics stream. The video stream represents video of a movie. The audio stream represents audio of the movie. The 
20 presentation graphics stream represents subtitles of the movie. The Interactive graphics stream represents dynamic 
control procedures for reproduction of menus. FIG. 4 schematically shows a structure of the AV Clip. 
[0021] The AV Clip shown in the figure on the middle level is obtained in the following manner, The video stream 
shown on the upper first level is composed of a plurality of video frames (picture pj1 , p]2, pJ3, ...), and the audio stream 
also shown on the upper first level is composed of a plurality of audio frames. The video and audio streams are sep- 

6 arately converted to PES packets shown on the upper second level. The PES packets are further converted to TS 
packets shown oh the upper third level: Similarly, the presentation graphics stream and the interactive graphics stream 
shown on the lower first level are separately converted to PES packets shown on the lower second level, and further 
converted to TS packets shown oh the lower third level. These TS packets shown on the upper and lower thirc} levels 
are multiplexed to form the AV Clip. 

30 [0022] The AV Clip generated as above is divided into a plurality of extents in the same way as computer files, and 
recorded onto the BD-ROM. The AV Clip Is composed of one or more Access Units, and the reproduction of AV Clip 
may skip to a point corresponding to ant Access Unit. An Access Unit is composed of one GOP (Group Of Pictures) 
and an audio frame to be read simultaneously with the GOP, and is a minimum unit for decoding. GOPs Include three 
types of pictures; Bldirectionally Predictive picture (B picture) that is coded using correlation with both past ancj future 

35 pictures; Predictive picture (P picture) that Is coded using correlation with past pictures; and Intra picture (1 picture) 
that is coded using Its own spatial frequency characteristic without referencing the inter-frame correlation. 
[0023] Clip information (XXX.CLPI) Is management Information of each AV Clip. Fig. 5 shows an internal structure 
of Clip Information. Since an AV Clip Is obtained by multiplexing a video stream and an audio stream and accessible 
at random in the units caltedAccess Units, the Clip information includes information, such as the attributes of video 

40 and audio streams and the points within the AV Clip to which a skip operation may be performed. In the figure, the 
doted tines Indicate that the structure of Clip Information is excerpted to be shown in detail. As indicated by th* doted 
lines hn1 , the Clip Information (XXX.CLPI) Is composed of "attribute information 11 of the video and audio strearjis, and . 
"EP_map" that is a reference table for searching Access Units. 

[0024] As doted lines hn2 indicate, the attribute information is composed ol an attribute of the video stream (Video 
45 Attribute), the number of attributes (Number), and attributes (Audio Attribute #1 -#m) of all the audio streams multiplexed 
in the AV Clip. As doted lines hn3 indicate, the video attribute Is composed of information showing the coding method 
in which the video stream is coded (Coding), the resolution of pictures constituting the video stream (Resolution), the 
aspect ratio (Aspect), and the frame rate (Framerate). ! 
[0025] As doted lines hn4 indicate, each of the attributes of audio streams (Audio Attribute #1 -#m) is composed of 
so information showing the coding method in which a corresponding audio stream is coded (Coding), the channel number 
of the audio stream (Ch.), the language of the audio stream (Lang.), and the sampling frequency. 
[0026] The EP_map Is a reference table for indirectly referencing by time the addresses of accessible points. In skip 
operations. As doted lines hn& indicate, the EP_map Is composed of a plurality of entries (Access Unit #1 entry, Access 
Unit #2 entry, Access Unit #3 entry ...), and the number of entries (Number). As doted lines hn6 Indicate, each entry 
55 shows the reproduction start time and the address of a corresponding Access Unit (Note that the size of a first I picture 
in the Access. Unit (l-slze) maybe additionally shown) . The reproduction start time is shown by a time stamp for a 
picture located at the beginning of the Access Unit (Presentation Time Stamp). The address is shown by a serial number 
of a corresponding TS packet (SPN: Source Packet Number). Due to the variable-length coding, the Access Units 
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each containing a GOP are not uniform both In size and reproduction time period. Yet, with reference to the entries 
corresponding to the Access Units, an Access Unit located at a point corresponding to any given reproduction time 
can be searched, so that the reproduction can be started from the first picture In the searched Access Unit. Note that 
"XXX M In the file name "XXX.CLPI" Is the same as the name of AV Clip to which the Clip information corresponds, in 

5 the figure, the file name of the AV Clip Is "XXX", which means that the Clip information (XXX.CLPI) corresponds to the 
AV Clip (XXX M2TS). This concludes the description of Clip information. Next, PlayUst Information will be described. 
[0027] YYY.MPLS (PlayLlBt Information) is a table serving as a PlayUst defining reproduction paths, and is composed 
of a plurality of pieces of Playltem information (Play Item Information #1, #2, #3,... #n) and the number of Ptayltem 
Information pieces (Number). FIG. 6 shows an internal structure of the PlayUst Information. PlayUst Information shows 

10 one or more logical segments for reproduction each defined by Playltem Information. As the doted lines hs1 Indicate, 
the structure of Playltem Information Is oxcerpted to be shown in dotal). As shown in the figure, the Playltem information 
is composed of: "olipjnformatlonjile.name" showing the file name of a reproduction segment within the AV Clip to 
which the injime and outjlme of the reproduction segment belong; "clip.codecjdentifler showing the coding method 
by which the AV Clip Is coded; "in^Ume" showing the time corresponding to the start point of the reproduction segment; 

is and °out_time" showing the time corresponding to the end point of the reproduction segment. 

[0028] One feature of the Ptayltem information ties in Its notatfonal conventions. That is, a reproduction segment Is 
defined by the Indirect reference by time, using the EPjmap as a reference table. FIG. 7 is a view schematically 
Illustrating the Indirect reference by time. In the figure, the AV Clip Is composed of a plurality of Access Units. The 
EP_map in the Clip Information specifies the sector address of each Access Unit as Indicated by arrows ay1 , ay2, ay3, 

20 anday4. Each of the arrows )y1 , jy2, jy3, and j y4 is a schematic representation of an indirect reference to an Access 
Unit, tn short, each piece of Playltem Information has a reference (arrows Jy 1 , jy2, Jy3, and Jy4) specifying by time via 
the EP.map, the address of a corresponding Access Unit contained in the AV Clip. 

[0029] A reproduction segment on the. BD-ROM specified by a set of Playltem information - Clip Information - AV 
Clip Is called "Playltem - . A logical unit of reproduction on the BD-ROM specified by a set of PL Information * Clip 
25 information - AV Clip is called "PlayUst (hereinafter, PL)", A movie recorded on the BD-ROM Is segmented into the 
logical units of PLs. Since the movie Is segmented Into the logical units, it is possible to define such a PL specifying 
scenes lhall of which a specific character makes an appearance. In this way, separately from the main movie, another 
movie Is readily made in which the character appears at all times. 

[0030] Since movies recorded on BD-ROM have the logical structure as described above, an AV Clip of a specific 
30 scene in a movie may be readily "recycled" or used In another movie. 

[0031] Next, a description Is given to an interactive graphical stream. FIG. 6A is a view showing a structure; of the 
graphical stream. On the first level, a string of TS packets constituting an AV Clip Is shown. On the second level, a 
string of PES packets constituting a graphics stream is shown. The PES packets shown on the second level are formed 
by concatenating payloads of TS packets having a predetermined PID within the TS packet string shown on the first 

6 level. Note that no description Is given to a presentation graphics stream because ft Is not the gist of the present 
invention. 

[0032] On the third level, a structure of the graphics stream Is shown. The graphics stream is composed of functional 
segments that include an ICS (Interactive Composition Segment), a PDS (Palette Definition Segment), an ODS (Object 
Definition Segment), and an END (End of Display Set Segment). Of these functional segments, the ICS Is a 'screen 

40 composition segment, whereas the PDS, ODS, and END are definition segments. Each functional segment is either 
In one-to-one or one-to-multlple correspondence with PES packets. That Is, one functional segment Is recorded on the 
BD-ROM 1 00 after being converted to a single PES packet, or fragmented and converted to a plurality of PES packets. 
[0033] FIG. 88 is a view showing the PES packets obtained by converting functional segment (s). As shown In FIG. 
88, each PES packet is composed of a packet header and a payload. The payload Is an entity of a functional segment, 

<5 and the packet header contains a DTS and PTS associated with that functional segment. Hereinafter, the DTS and 
PDS contained tn the header of a PES packet containing a functional segment are referred to as the DTS and; PTS of 
that functional segment. 

[0034] These various types of functional segments define a logical structure as shown in FIG. 9. In the figure, the 
functional segments are shown on the third level, display sets are shown on the second level, and Epochs are shown 
so on the first level. 

[0035] Each display set (herein after "DS") shown on the second level is a group of functional segments which tpgethe r 
constitute one complete screen of graphics. Dashed lines hk2 Indicate a DS to which the functional segment^ on the. 
third level belong. As can be seen from the figure, the series of functional segments, ICS-PDS-ODS-END constitutes 
one DS. Upon reading these functional segments constituting the DS from the BD-ROM, the reproduction apparatus 
ss can produce one screen of graphics. 1 

[0036] An Epoch shown on the first level refers to a time period during which the continuity in memory management 
must be maintained on the reproduction time axis of AV Clip, or to a set of data allocated to the time period, fernery 
mentioned here Includes a graphics plane for storing one screen of graphics and an object buffer for storing jpneom- 



BttSOOCIO. <EP l608ie&A1J_> 



EP1 608 165 A1 

pressed graphics data. The continuous memory management means that throughout the Epoch neither the graphics 
plane nor the object buffer is flushed, and erasing and rendering ol graphics are performed only within a predetermined 
rectangular area of the graphics plane (to flush means to clear the entire graphics plane and the entire object buffer). 
The size and position of this rectangular area are fixed throughout the Epoch, So long as erasing and rendering of 
graphics are performed within this fixed rectangular area of the graphics plane, seamless reproduction Is ensured. In 
other words, the Epoch is a time unit, oh the reproduction time axis of the AV Clip, during which seamless reproduction 
is ensured. To change the graphics rendering area in the graphics plane, it is necessary to define a point of cfiange 
on the reproduction time f xis and set a new Epoch from the point onward. In this case, a boundary between the two 
Epochs Is not seamless. 

'<> [0037] The seamless reproduction used herein means that the eraslng/renderlng of graphics is completed with a 
predetermined number of video framns. In the case of interactive graphics stream, the number of video frames is four 
to five. Trie number of video frames is determined based on the ratio of the fixed region to the entire graphics plane 
and the transfer rate between the object buffer and the graphics plane. 

[0036] In the figure, dashed lines hk1 and hk2 Indicate an Epoch to which the functional segments shown on the 
15 second level belong. It is shown that a series of DSs, which are an Epoch Start DS, an Acquisition Point OS t and a 
Normal Case DS, constitutes one Epoch on shown on the first level. Here. "Epoch Start", "Acquisition Point 1 , and 
"Normal Case" are types of DSs. Although the Acquisition Point DS precedes the Normal Case DS in FIG. 9, they may 
| be arranged in the reverse order. 

; [0039] The "Epoch Start" DS provides a display effect "new display", and indicates a start of a new Epoch, The Epoch. 

) 20 start DS thus contains all functional segments needed for the next screen composition . The Epoch Start DS Is provided 
j at a point in the AV Clip to which a skip operation may be performed, such as a start of a chapter in a movie. 

! [0040] The Acquisition Point DS provides a display effect "display refresh", and relates to the preceding Epoch Start . 

] DS. There are two types of Acquisition Point DS: "Duplicate" and "Inherit", The Duplicate type DS is a completely 

| Identical DS to the preceding Epoch Start DS. The Inherit DS inherits the functional segments of the preceding Epoch 

» start DS but has different button commands. Although not being a start of an Epoch, the Acquisition Point DS contains 
i _ all functional segments needed for the next screen composition. Thus, when the reproduction is started from pn Ac- 

) * quisftion Point DS, the graphics are reliably presented. That is to say, the Acquisition Point DS enables a screen 

\ composition from a midpoint in the Epoch. 

[0041] The Acquisition Point DS fs provided in a point to which a skip operation may be made. Examples of such 
points include one that may be designated by a time search. The time search Is an operation of locating a reproduction 
) point corresponding to a time inputted by a user in minutes/seconds. Since the user Input is made in a relatively targe 

| unit such as ten minutes and ten seconds, searchable reproduction points are located at intervals of 1 0 minutep or 1 0 

\ seconds. By providing the Acquisition Point DS in such a point searchable by a time search, the graphics stream can 

j be smoothly presented when a time search Is conducted. 

■ . 35 [0042] The Normal Case DS provides a display effect "display update", and contains only a difference from the 
j previous screen composition. For example, if a button defined by a DS (v) has the same graphical representation as 

i a button defined by an Immediately preceding DS (u) t but has a different state control. In this case, the DS (v) contains 

{ ICS or ICS and PDS only, and serves as a Normal Case DS. With this arrangement, the Normal Case DS is not required 

to contain overlapping 6 DSs, which leads to the reduction In the amount of data stored on the BD-ROM. Sirjce the 
40 Normal Case DS only contains the difference, graphics cannot be displayed with the Normal Case DS alone. \ 

[0043] The interactive display defined by the above-described DSs presents GUI elements. The interactivity pf DSs 
| refers to the capability of changing the state of each QUI element in accordance with a user operation. In this embod- 

! iment, GUI elements interacted by user operations are referred to as buttons. Each button has the normal st^te, the 

I< selected state, and the activated state. Each button state Is rendered using a plurality of pieces of uncomppssed 

49 graphics data, which is referred to as a "graphics object" . One state of one button is associated with a plurality of 
graphics objects for presentation in animation. 
( [0044] Now, a description is given to Definition Segments (ODS: Object Definition Segment and PDS: Palette Defi- 

nition Segment). 

[0045] The ODS Is information defining a graphics object, which will be described later. Since AV Clips recorded on 
so a BD-ROM feature its high Image quality comparable to high-definition television, the graphics objects have a high 
resolution of 1920 x 1080 pixels. The color of each pixel fs defined by 8-bit long index values showing a red color 
difference component (Cr value), a blue color difference component (Cb value), a luminance component (Y value), 
and a transparency (T value). This structure allows each pixel to be set to one of arbitrary 256 colors out of 1 6,777,21 6 
colors. 

ss [0046] The ODS has a data structure shown in FIG. 10 A. As shown in the figure, the ODS Is composed of the following 
fields: "segment_type" showing that the type of this segment is an ODS; "segment Jength' showing the data length of 
the ODS; 'objecUd" uniquely identifying a graphics object within the Epoch, associated with the ODS; 
"object_ver8lon_number l, showing the version of the ODS within the Epoch; ■lasUn_sequence_flag"; and 
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"obJecLdataJragment" carrying a consecutive sequence of bytes corresponding to part or all of the graphics object. 
[0047] The object Jd field uniquely identifies a graphics object within the Epoch, associated with the ODS. In the 
case where a plurality of graphics objects defined by a plurality of ODSs constitutes a sequence of animation, these 
ODSs are assigned serial objectjd values. 

5 [0040] Referring now to the last Jn_sequencejlag field and the object_data_fragment field In more detail. Due to 
the constraint on a payload of PES packet, there may be a case where a piece of uncompressed graphics data con- 
stituting one button can not be carried by a single ODS. If this is the case, the graphics data is fragmented and each 
fragment is defined by an ODS in the ob|ect_data_fragment field. Here, every fragment except the last fragment Is of 
the same size. That is , the last fragment Is {ess than or equal to the size of the preceding fragments. The ODSs oarrytng 

to these fragments of the graphics object appear in the DS in sequence. The lastjn.sequence Jlag field indicates an 
end of the graphics object. Although the above ODS data structure is based on a method of storing fragments in 
consecutive PES packets without a gap, the fragments may instead be stored In PES packets so as to leave some 
gaps between the PES packets. This concludes the description of ODS. 

[0049] Next, the PDS Is explained. The PDS Is Information defining a palette for color conversion. FIG. 108 shows 
15 a data structure of the PDS. As shown In the figure, each PDS is composed of the following fields: " segment Jype" 
showing, when set to the value "0x15", that the type of this segment is a PDS; "segmentJength* showing the data 
length of the PDS; 'pallet Jd" uniquely identifying the pallet contained in the PDS; "pa left e_ version .number" showing 
the version of the PDS within the Epoch; and "palleLentry" showing the Color Difference Red (Cr.value). the Color 
Difference Blue (Cb_value), Luminance (Y_value), and Transparency (T_value). 
so [0080] Next, an ICS is explained. The ICS Is a functional segment defining the composition of an interactive display. 
The ICS has a data structure shown In FIG. 11 . As shown In the figure, the ICS Is composed of the following fields, 
•segmenOype", "segmentJength", n composttlon_number*, "composltlon.state", w command_updateJlag", 
■cornpositionjlme.outjrts", l, selectton_tlme_out_pts\ "U0 - maskJable , \ "anlmationJrame_rate^code n ( 
"default_selected_button_number, ■default_activated_button_number" p and a group of "button info (1), (2), (3)". 

6 [0051] The composttton_number field Is set to a value out of 0-15 showing that the DS to which the ICS belongs Is 
updated. 

[0052] The compositlon_state field shows that the DS beginning with the ICS is the Normal case DS, the Acquisition 
Point DS, or the Epoch Start DS. 

[0053] The command_updateJlag field Indicates that the button commands in the ICS are changed from those 
so defined in the previous ICS. For example, the DS to which an ICS belongs is an Acquisition Point DS, the ICS Is 
normally equal to those defined in the immediately previous ICS. Yet, by setting the command.update Jlag to ON , the 
ICS can define different burton commands from those defined in the immediately previous ICS. The 
command.update Jtag is set to ON when different button commands are associated with the same graphics objects. 
[0054] The composition jlme_outj)te field describes the end time of the Interactive display. At the end tlrne, the 
& Interactive composition is no longer valid and thus no longer displayed. The composition Jlme.ouLpls Is preferably 
described with the frame accuracy of the video stream on the reproduction time axis. ( 
[0055] The selection_tlme.out.pt8 field describes the termination time of the valid button selection period. At the 
time of the selectlonjime_out_pts, the button specified by the defautt_actlvated_button_number Is activated. The 
value of the selectlon.tlme.outjrts is less than or equal to the value of the composltlon_tlme_out_pte, The 
40 selection Jime„out_pts Is described with the frame accuracy of the video stream on the reproduction time axis. 

[0056] The UOjmask jable determines permissions /prohibitions of User Operations for the duration of DS to which 
the ICS belongs. When this field is set to u Prohibited", a corresponding User Operation to the reproduction apparatus 
is Invalid. 

[0057] The animation Jrame_rate_code field specifies the frame rate to be applied to animated buttons. The anlma- 
45 tlon frame rate is given by the video frame rate divided by the value of the animation J rame_rate_code field. When 
this field is set to "00", only the graphics object designated by the start_objectJd_xxx is displayed for every button in 
non-animated state. The default_selected_button_ number field Indicates the button number that is selected as default 
when the presentation of the interactive display begins. When this field is set to the value "0", the button specified by 
the button number stored in a register of the reproduction apparatus Is automatically activated. On the other hand, 
90 when this field is set to a value other than "O", the value Indicates a valid button number. 

[0058] The default_acttvated_button_number field Indicates the button which Is automatically activated when no 
button is activated by the user before the time defined by the selection jlme_out_pte field. When this field Is set to 
"FF\ the currently select button Is activated at the time defined by the selection Jlme.out jptB field, When thlj field is 
set to "00", no button is automatically activated. If this field is set to a value other than "FF" and "00", the value Is 
55 interpreted as a valid button number. 

[0059] The buttonjnfo field provides Information defining buttons to be presented on the Interactive display. In the 
figure, the doted lines hp1 Indicate that a data structure of buttonjnfo (I) Is excerpted to be shown In detail. The 
buttonjnfo (i) carries Information about the button (I) defined by the ICS. Hereinafter, a description Is given to the 
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information items constituting th© buttonjnfo (i). 

[0060] The button_number field indcates the value uniquely identifying the button (i) within the ICS. 
[0061] The numerical ry_setectablejlag field indicates whether the button (I) can be numerically selected. 
[0062] The auto.action Jlag field indicates whether to automatically activate the button (i) . When the auto.action Jlag 
9 is set to ON (bit value of "1"), the button (I) Is transferred not to the selected state but directly to the activated state. 
On the other hand, when the auto.actionjlag is set to OFF (bit value of "0"), the button (I) is transferred hot to the 
activated state but lo the selected state when selected by a user. 

[0063] The button_horizontal_posltton field and button_verticai_posrtion field respectively specify the horizontal and 
vertical positions of the top left pixel of the button (i) in the interactive display. 
to [0064] The upperJ>utton_number field specifies the button number of a button to receive the selected state at a 
push of the Move Up key while the button (I) is in the selected state! If this field Is set to the same button number as 
that of the button (I), the user operation to the Move Up key Is ignored. 

[0065] Similarly, the lower_button_number field, the left J>utton_number field, the right _button_number field specify 
the button number of a button to receive the setected state at a push of the Move Down key, Move Left key, and Move 
?5 Right key, respectively, while the button (i) is in the selected state. If these fields are set to the same button number 
as that of the button (i), the user operations to the respective keys are ignored. 

[0066] The start^objectj cLnormal specifies the first one of object Jds assigned serially to a group of ODSs consti- 
tuting the animation of button (i) in the normal state. 

[0067] The end__objectjd_normal field specifies the last one of the objecUds assigned serially to the group of ODSs 
so constituting the animation of button (I) in the normal state. If the end_objectjd_normal field specifies the same value 
as that of the start_ob]ect_ld_normal, the static image of the graphics object identified by the value is presented as the 
button (i), 

[0066] The repeat_normal_flag field specifies whether the animation of the button (I) in the normal state is to be 
continuously repeated. 

25 [0069] The start_ob)ectJd_selected field specifies the first one of objectjds assigned serially to a group of ODSs 
constituting the animation of button (I) in the selected state. If the end_ob]ectJd_seiected field specifies the same 
value as that of the 8tarLobJectJd_selected, the static image of the graphics object identified by the value Is presented 
as the button (I). 

[0070] me end_ob]ectJd_selected field specifies the last one of the objectjds assigned serially to the group of 
30 ODSs constituting the animation of button (i) in the selected state. 

[0071 ] The repeat_selected_1lag" field specifies whether the animation of the button (I) in the selected state is to be 
continuously repeated. 

[0072] The start_obJect JcLactlvated field specifies the first one of objectjds assigned serially to a group of ODSs 
constituting the animation of button (I) in the activated state. 
35 [0073] The end_object_id_activated field specifies the last one of the objectjds assigned serially to the group of 
ODSs constituting the animation of button (i) in the activated state. 
[0074] Next, a description is given to button commands. 
[0075] Each button.command is executed when the button (i) is activated. 

[0076] The button commands may include one Instructing the reproduction apparatus to reproduce a PL or Playltem. 
to Such a command instructing the reproduction apparatus to reproduce a PL and Playltem Is referred to as a LinkPL 
command. When this command is executed, reproduction of the PlayLlst specified by the first argument is started from 
the point specified by the second argument. 



45 



Format: LinkPL (first argument, second argument) 



■\ ' [0077] The first argument is the PL number specifying the PL to be reproduced. The second argument specifies, as 

| a reproduction start point, a Playltem, a Chapter, or a Mark within the PL. 

\ [0078] The LinkPL function for specifying a Playltem as a reproduction start point is "LinkPL at Playltem ()". 

j [0079] The LinkPL function for specifying a chapter as a reproduction start point is "LinkPL at Chapter ()*. 

] 50 [0080] The LinkPL function for specifying a mark as a reproduction start point is "LinkPL at Mark ()". 

* [0081] The button commands may also include a command Instructing the reproduction apparatus to obtain or set 
i ' the state of the apparatus. The state of reproduction apparatus is shown by 64 player status registers (whose values 
\ . are ref erredtoasPSRs) and 4, 096 general purpose registers (whose values are referred to as GPRs) . With the following 
; commands (I) - (Iv), specific values are set to the registers or the values of the registers are obtained. 

* 55 
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(I) Get value of Player Status Register Command 
Format: Get value of Player Status Register (argument) 
s [0062] This function returns a value of the player status register specified with the argument, 
(il) Set Value to Player Status Register Command 

Format: Set value of Player Status Register (first argument, second argument) 

10 

[0063] This function sets a value of tho player status register specified with the first argument to the second argument, 
(ill) Get value of General Purpose Register Command 
15 Format: Get value of General Purpose Register (argument) 

[0084] This function retunes a value of the general purpose register specified with the argument, 
(iv) Set value to General Purpose Register Command 

20 

Format: Set value to General Purpose Register (argument) 

[0066] This function seta a value of the general purpose register specified with the first argument to the second 
argument. 

25 [0086] This concludes the description of the ICS structure. Next a description is given to a specific example of 
interactive control defined by the ICS. The example relates to the ODSs and ICS shown in FIG. 1 2. FIG. 12 Is a view 
showing the relation between the ODSs and ICS contained within a OS (n). The DS (n) includes the ODSs 11-19, 
21-29, 31-39, 41-49. The ODSs 11-19 are for rendering a button A in each of the three states. The ODSs 21-29 are 
for rendering a button B in each of the three states. The ODSs 31-39 are for rendering a button C in each of the three 

30 states. The ODSs 41-49 are for rendering a button O in each of the three states (See the right brackets In the figure). 
The buttonjnfo (1 ), (2), (3), and (4) in the ICS provide the descriptions of state control of the buttons A-D (See arrows 
bh1 , bh2, bh3, and bh4 in the figure). 

[0087] Suppose that the control defined by the ICS Is to be executed In synchronism with the display timing of a 
picture pti included in the video stream shown In FIG. 13. in this case, an Interactive display tm1 composed of the 
as buttons A-D is overlaid (gs1 ) with the picture pt1 to produce a composite screen gs2. In this way, the Interactive display 
composed of multiple buttons is presented In synchronism with a specific video Image. Thus, the ICS makes it possible 
to present buttons In a way more real to the users. 

[0066] FIG. 1 4 shows an example of the description realizing the state transition of buttons A-D as Illustrated In FIG. 
15. In FIG. 15, the arrows hh1 and hh2 each visually represent the state transition defined by the neighbojjnfo () 

40 within tho button into (1 ). In the neighborjnfo (), tho lower J>utton_number field specifies the button C. Consequently, 
a user operation to the Move Down key (up1 in the figure) while the button A Is in the selected state, the fcjutton C 
receives the selected state (sj1 in the figure). Similarly, the rightJ>utton_number field specifies the button B.'Conse- 
quentty, a user operation to the Move Right key (up2 in the figure) while the button A is in the selected state, the button 
B receives the selected state (sJ2 in the figure). 

45 [0089] The arrow hh3 in FIG. 15 visually represent the state transit ton defined by the neighborjnfo {) within the 
button Info (3). In the neighborjnfo 0. the upper J>utton_number field specifies the button A. Consequently, a user 
operation to the Move Up key (up3 in the figure) while the button C is in the selected state, the button A is put back 
into the selected state, ; 
[0090] Next, a description is given to graphical representations of the buttons A-D. The ODSs 1 1 , 21 , 31 , and 41 

so each present images shown in FIG. 16. The ODSs 11-19 associated with the button A present images shown In FIG. 
17. Thestart_ob]ectJd normal and end_ob]ectJd normal fields specify the ODSs 11 and 13, respectively. Consequent- 
ly, the button A in the normal state is presented in animation by the sequence of the ODSs 11-13. In addltlop, In the 
selected_statejnfo ().of the buttonjnfo (1), the start_ob]ectJd„selected and end_obJectJd_eelected fields specify 
the ODSs 14 and 16, respectively. Consequently, the button A In the selected state is presented In animation by the 

55 sequence of ODSs 14-1 6. When a user puts the button A Into the selected state, the Images presented for thp button 
A is changed from the ODS 11-13 to the ODSs 14-16. By setting the repeat_normalJlag and repeat^selectedjlag to 
the value "1", the animation of the ODSb 11-13 and of the ODSs 14-16 are continuously repeated, as represented In 
the figure with A", "A-V, B" and "B ' 
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[0091 J As described above, each of the buttons A T Bfe associated with of a sequence "of ODSs which can be presented 
in animation. Thus, with the ICS describing the control as above, such button state control is realized that a character 
image serving as a button changes its facial expression In response to user operations. 

[0092] Next, a description is given to application of the_numerically__selectable_fiag field. FIG. 18 is a view showing 

.5 an example of the ICS and ODSs Included In a DS. The ODSs 31-33 represents images of .three baseball players 
along with their names and ptayer numbers, as shown In the upper part of the figure. The ICS belonging to the DS 
includes three pieces of button information, buttonjnfo (1), (2), and (3). The start_objectJd fields of. buttonjnfo (1), 
(2), and (3) specify the ODS 31, ODS 32, and ODS 33, respectively. Similarly, the button .number fields of the 
buttonjnfo (1), (2), and (3) are set to the values of "99", "42", and "94° , respectively. Here, the 

10 numerically_eetectablejlag fields of the buttonjnfo (1), (2), and (3) are all set to the value "f. With the above ar- 
. mngemeht, each button defined by buttonjnfo (1), (2), and (3) Is numerically selectable. When a user inputs the value 
W on the remote controller 400, the button presented with the image of Mr. Beginner's Luck receives the selected 
state. The value 9 99° may be inputted by pressing the "4" key and "?9" key in a row or by pressing the "9" key owe and 
"+1 0" key four time in a row. Alternatively, when a user Inputs the value *42 H , the button of Mr. Careless Mistake receives 

15 the selected state. When a user inputs the value w 94 i , the button of Mr. Dead Stock receives the selected state. 

[0093] Here, suppose the auto^action.flag fields of the buttonjnfo (1), (2), and (3) are.set to the value "1". In this 
case, the above three buttons receive, when selected, the active state instead of the selected state, and the button 
commands (LinkPL (PL#21), UlnkPL (PL#22), LinkPL (PL#23)) included in the buttonjnfo are executed. As a result, 
PL #21 , #22, or #23 to which the executed button command is linked is reproduced. If those PLs define scenes of 

20 batting or pitching by the above piayers.'each of these scenes is reproduced at an input of the numeric value* corre- 
sponding to one of the player numbers. Since the buttons are selectable with an input of the player numbers, which 
are easy for the users to recognize, there Is provided improved user operabittty. 

[0094] Next, a description is given to the order of ODSs in the DS. As already described above, the ODSs be^nglng 
to one OS are associated by the ICS with each state of one button. The order of ODSs in the DS is determined In 

23 accordance with the button state with which each ODS Is associated. 

[0095] In more detail, the ODSs belonging to one DS are grouped into (1) ODSs for. presentation of a button in the 
normal State, (2) ODSs for presentation of the button in the selected state, and (3) ODSs for presentation of the^ button 
in the activated state. Each group for presentation of a respective state is referred to as a "button-state group". The 
button-state groups are arranged in the order, for example, of the normat state -» the selected state -> the activated 

30 state. As above, the order of ODSs within the DS is determined according to the button state with which the OE^Ss are 
associated. 

[0096] FIG. 1 9 is a view showing the order of ODSs within a DS. in the figure, the following three button-state groups 
within the DS are shown on the second lewel: the button state group for presentation of the normal state (ODSs for 
Normal state) ; the button state group for presentation of the selected state (ODSs for Selected state) ; and th^j button 
35 state group for presentation of the activated state (ODSs for Activated state) . jn the figure, the button-state groups are 
arranged in the order of the norma! state -» the selected state -> the activated state. This order is determined so that 
the reproduction apparatus first reads the interactive composition constituting the first interactive display, and then 
reads the interactive composition to be presented only after update. 

[0097] FIG. 19 shows on the first level the graphics objects An; Bn, Cn, Dn, As, Bs, Cs, Ds, Aa, Ba, Ca, and Da that 
40 are referenced by the button-state groups. The numerical subscript n as In An, Bn, Cn, and Dn indicates a respective 
button In the normal state. Similarly, the numerical subscript s as In As, Bs, Cs, Ds indicates a respective button in the 
selected state, and the numerical subscript a as In Aa, Ba, Ca, and Da indicates a respective button in the activated 
state. The figure shows on the second level the button-state groups to which the graphics objects shown on pie first 
level.belong. In the figure, there are more than one ODS appendedwith the numerical subscripts, such as "1 ■ pnd" n" 
45 to read "ODS 1* and "ODS rf . Yet, it should be noted that each ODS 1 included In N-ODS, S-ODS andA-ODS is 
different. The same holds to the drawings with similar reference numerals. 

[0098] FIG. 20 is a view illustrating the button state transition on the Interactive display defined by the button-state 
groups shown in FIG. 19. 

[0099] As shown In the figure, the interactive display has a plurality of states Including "Initial state", "display update 
so by 1 st user action*, and "display update by 2nd user action". The arrows in the figure represent user actions triggering 
the state transition. The four buttons A, B, C, and D each have the normal state, the selected state, and the activated 
state. In order to present the initial interactive display, the graphics objects for the three buttons In the normal state 
and one button in the selected state need to be ready for presentation 

[0100] When a default selected button is not specified, it is not determined which of the buttons A-D will be selected 
55 first. Even so, the initial interactive display can be made upon completion of decoding the graphic objects for the s normal 
and selected states of each button. In view of this observation, the.button-state groups in this embodiment are ajrranged 
in the order of thenormal State ->.the selected state -> the activated state, as shown in FIG. 19. With this arrangement, 
the Initial interactive display can be presented even if the ODSs for the activated state are not yet read and decoded. 
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As a result, the duration is shortened between the reading of DS Is commenced and the presentation of Initial interactive 
display is completed. 

[0101] With reference to FIG. 21 , a description Is given to the order of ODSs that are shown In RGs. 1 6 and 17. FIG. 
21 is a view showing the order of ODSs within a DS. In the figure, the ODSs for normal state include ODSa 11-13, 

5 21-23, 31-33, and 41-43. The ODSs for selected state Include ODSs 14-16, 24-26, 34-36, and 44^46. The opSs for 
activated state Includes ODSs 17-19, 27-29, 37-39, and 47-49. The ODSs 11-13 are for animated prase ntatlop of the 
character image shown In FIG. 1 7. Similarly, the ODSs 21 -23, 31 -33, and 41 -43 are for animated presentation pt other 
character images.- Thus, by placing the button-state group composed of the above ODSs (ODS 11-13, 21-23, 31-33, 
41-43) as the first button state group in the DS, the initial Interactive display is ready for presentation even before the 

to reading of the DS is completed. This ensures that the interactive display composed of animated buttons is displayed 
without delay. 

[0102] Next, a description is given to the order of. ODSs when there are multiple references thereto from a plurality 
of button-state groups.. The term multiple references means that the same object Jd is referenced by two or more 
pieces of normaLstate Jnfo, selected.etatejnfo, and actlvated_state Jnf o within the ICS, With the multiple references, 
for example, a specific graphics object used to render a button In the normal state is common ty used to render another 
button in the selected state, in other words, the graphics object is shared, so that the number of ODSs can be reduced. 
Here, a problem arises as to which button-state group an ODS with multiple references belongs. Specifically, when 
one ODS Is associated with one button In the normal state as well as another button in the selected state, It needs to 
be determined whether the ODS belongs to the button-state group for the normal state or the button-state group for 

20 the selected state. In such a case, the ODS is placed in the button-state group that first appears. For example, If the 
button-state groups appear in the order of the normal state -» the selected state -> the activated state, an ODS refer- 
enced by both the normal-state and selected-state groups is placed in the normal-state group. Also, an ODS referenced 
by both the selected-state and activated-state groups is placed in the selected-state group. This concludes the de- 
scription of the order of multiple referenced ODSs. 

25 [0103] Now, a description Is given to the order of ODSs within the button-state group for the selected state. In the 
button-state group for the selected state, which of the ODSs should be placed at the beginning depends on whether 
a default selected button is determined. The default selected button is specified when the 
default_selected_button_number field Is set to a valid value other than "00". in this case, the ODS associated with the 
default selected button Is placed at the beginning of the button-state group. 

30 [01 04] No default selected button is specified when the default selected button_number field In ICS is set to th>e value 
"00". The defaulLselected_button_number field is set to the value "OCT in the case, for example, where an AV Clip Into 
which the DS is multiplexed serves as a merge point of a plurality ol reproduction paths. Suppose that the preceding 
reproduction paths correspond to Chapters 1 , 2, and 3, a DS Serving as a merge point Is for presentation of buttons 
associated with Chapters 1 , 2, and 3. In such a case, no specific- button can be specified in the 

35 defeult_8elected_button„number field, 

[0105] Ideally, when the Interactive display Is presented after reproduction of Chapter 1 , the button assoclajed with 
Chapter 2 should be selected as default. Similarly, after reproduction of Chapter 2, the button associated with phapter 
3 should be selected as default, and the button associated with Chapter 4 after reproduction of Chapter 3. jn other 
words, the defaulLselected J>utton_number field Is set to the value "0" to be Invalidated when a different buttop needs 

40 to be selected as default depending on which reproduction path has been taken. In this case, It is not necessary to 
. place a specific button-state group ODSs at the beginning because no specific button is selected as default. ' 
[0106] This concludes the description of the order of ODSs. Next, a description is given to how DSs having above- 
described ICSs and ODSs are allocated on the reproduction time axis of AV Clip. An Epoch Is a time period on {he 
reproduction time axis during which memory management Is continuous. Since each Epoch Is composed of one or 

45 more DSs, It Is Important to how to allocate those DSs on the reproduction time axis of AV Clip. The reproduction time 
axis of AV Clip mentioned here Is a time axis for defining decoding timing and presentation timing of individual pictures 
which constitute the video stream multiplexed In the AV Clip. The. decoding timing and presentation timing on the 
reproduction time axis are expressed with a time accuracy of 90 KHz. This time accuracy of 90 KHz corresponds to a 
common muitlpieof NTSC signal, PAL signal, Dolby AC-3, and the frame frequency of MPEG audio. A DTS and a PTS 

50 attached to ICS and ODS within a DS specify timings on this reproduction time axis for achieving synchronous-control. 
In other words, the DSs are allocated on the reproduction time axis by exercising synchronous control using DTS and 
PTS attached to ICS and ODS. 

[0107] First, a description is given to the synchronous control exercised using DTS and PTS of an ODS. 
[0108] The DTS shows a time, with the time accuracy of 90 KHz, at which the decoding of ODS needs to be started. 
55 The PTS shows a deadline for completing the decoding. 

[0109] The decoding of ODS can not be completed Instantly, and takes a certain duration. In order to Qxpltclfly show 
the start and end times of decoding the ODS, the DTS and PTS of the ODS show the decoding start time and decoding 
deadline. 
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[0110] The value of PTS shows the deadline by which the decoding of ODS (J) needs to be completed and the 
resulting uncompressed graphics object needs to be available In the object buffer of the reproduction apparatus 
pi 1 1 ] The decoding start time of an arbttraryODS (J) belonging to a DS (n) is shown by DTS (DSn [ODS]) with the 
time accuracy of 90 KHz. Thus, the decoding deadline of the ODS (j) is determined by adding to the DTS (DSn [ODS]) 
5 value, a maximum time which may be taken for the decoding. 

[0112] Lei SIZE (DS/(ODS/|) denote the size of ODS (j), and Rd denote the ODS decoding rate. Then the maximum 
time required for the decoding (in seconds) is SIZE(DSn[ODS/))//Rd. 

[0113] By converting this maximum time to the accuracy of 90 KHz and adding the result to the DTS of the ODS (|). 
the decoding deadline to be shown by the PTS Is calculated with the accuracy of 90 KHz. 
10 [01 1 4] The PTS of the ODS (j) belonging to the DS (n) can be expressed by the following formula: 

PTS (DS[ODS/J) = DTS(DSn [ODS/]) + 
15 90,000 x (SlZE(DSn[ODSj])// Rd) 

[0115] Next a description is given to the value of PTS within an ICS. 

[01 16] The PTS within the ICS Is set to the value obtained by adding (1 ) the PTS value of the ODS of which decoding 
time Is latest among ail the ODSs needed for the initial display of the DS (n), (2) the time taken to clear the graphics 

20 plane, and (3) the time taken to transfer the decoded graphics object to the graphics plane. 

[0117] When the default_seiected_button_number is specified in the ICS, the initial interactive displayed can be 
presented upon completion of the decoding of ODSs associated with each button in the normal state and the specified 
default selected button in the selected state. The ODSs for rendering each button In the selected state are referred to 
as S-ODSs, and the S-ODS of which decoding start time Is earliest (i.e. the ODS for rendering the default selected 
button, in this case) Is referred to as S-ODSsflrst. The PTS value of this S-ODSsfirst is designated as the PT$ value 
of the ODS of which decoding start time Is latest, and used as a reference value for the PTS within the ICS. 
[0118] When the defauft_selected_button_number Is not specified In the ICS, It is not known which of the tjuttons 
wiltbeftrstselected. Thus.thelnitialinteractive display is not available for presentation until it is ready to render every 
button in the normal state as well as the selected state. Among the S-ODSs for rendering the selected state 9! each 

ao button included in the In Itlal lnteractive display, the 1 ODS of which decoding start time is latest is referred to as the 
S-ODSslast. The PTS value of this S-ODSslast Is designated as the PTS value of the ODS of which decode start time 
Is latest; and used as a reference value for the PTS within the ICS. 

[01 19] Let PTS (DSn[S-ODSsflrst)] denote the deciding deadline of S-ODSsflrst, then PTS (DSr(ICS]) Is set to the 
value obtained by (2) the time taken to clear the graphic plane and (3) the time taken for rendering the decoded graphics 

35 object to the graphics plane; 

[01 20] Let vtdeo.width and video Jieight denote the width and height of a rectangular area of graphics plane within 
which graphics objects are rendered. When the rendering rate to the graphics plane is 128 Mbps, the time t^ken to 
clear the graphics plane is obtained by 8 x video_width x vldeo_height // 128,000,000. With the time accuracy of 90 
KHz, the above-mentioned time (2) taken to clear the graphics plane is expressed by 90,000 x (8 x video_width x 

40 video_height // 128, 000, 000) . . . ' i" 

[0121] Let ZSIZE(DSn[ICS;BUTTON[q] denote the total size of graphics objects specified by each piece of button 
Information contained in the ICS. When the rendering rate to the graphics plane is 1 28 Mbps, the time taken to render 
the graphics plane is obtained by XSIZE(DS/i[ICS.BUTTON[/j]) // 128,000,000. With the time accuracy of 90 KHz, the 
above-mentioned time (3) taken to render the graphics plane is expressed by 90, 000 x (SSIZE(DSn{ICS. BUTTON 

45 H])// 128,000,000). 

[0122] With the above formulas, PTS (DSn [ICS]) Is expressed by the following formula: 

PTS(DSn[ICS]) 2 PTS (DSn [S-ODSsflrst]) + 
50 90,000 x (8 x video_width x videojieight // 128, 000, 000) 

+ 90,000 x (ISIZE(PSn[ICS.BUTTON[/]]) // 128,000,000) 

55 [0123] Note that the above formula is applicable when the defautLselectedJ)utton_number in the ICS is valid. If 
not, the following formula needs to be satisfied. 
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PTS(DSnflCSl) * PTS(DSn[S^DSslastD + 
90,000 x (8 x video.wldth x vldeo_helght//128,000,000) + 
5 90,000 x (£SI2E(DSn[ICS.BLnrTON[/l]y/128.000,000) 

[0124] By setting the PTS and DTS In the above manner, synchronous display is realized. FIG. 22 illustrates an 
example of such control. In this example, buttons are to be displayed in synchronization with the display timing of a 
to picture pyl In the video stream. To this end, the PTS within the ICS needs to be set to the value corresponding to the 
display point or picture data py1 . Note that the DTS wtthln the ICS Is set to show the time prior to the time shown by 
the PTS. 

[0125] The decoding of ODSs constituting the initial interactive display of DS (n) needs to be completed by the time 
calculated by subtracting from the time shown by the PTS In the ICS, the plane clear duration cd1 and the object 

» transfer duration td1 . Thus, the PTS of. the ODS of which decoding time is the latest needs to be set to the value 
corresponding to a point of time marked in the figure with a black star. Further, a duration dd1 is taken for decoding 
the ODS, so that the DTS of the ODS needs to be set to a time earlier than the PTS by the duration dd1 . 
[0126] The example illustrated in FIG. 22 is a simplified case In that only one ODS is used for overlay with the video 
data. In the case where a plurality of ODSs are used for presentation of the Initial interactive display, the PTS and DTS 

so of the ICS and of ODS need to be set as illustrated in FIG. 23. 

[0127] FIG. 23 is a view Illustrating the DTS and PTS setting in the case where the initial interactive display is con- 
structed by a plurality of ODSs and a default selected button is specified. Suppose that the duration dd1 is taken for 
decoding S-ODSsfirst, which is the ODS of which decoding time is latest among all ODSs needed for presentation of 
the initial interactive display. The PTS (DSnfS-ODSsfirst]) of this S-ODSsfirst is set to the value corresponding to the 

ss end of the duration ddl . 

[01 28] Further, for presentation of the initial interactive display, the graphics plane must be cleared and the decoded 
graphics objects must be transferred. Thus, the PTS (DSnflCSj) of the ICS must be set to the value corresponding to 
the time, at the earl est, calculated by adding to the value of PTS(DSn[S-ODSeflr8t]), the plane clear duration (90,000 
x (8 x video_wldth x video_height//1 28,000,000) and the decoded object transfer duration (90,000 x (2SI2E(DSn 

30 [ICS.BUTTON[(J))// 128,000,000)). 4 

[0129] FIG. 24 Is a view illustrating the DTS and PTS setting in the case where the Initial interactive display is con- 
structed by a plurality of ODSs and no default selected button Is specified. Here, the PTS (DSn [S-ODSslast]) is set 
to the value corresponding to an end of a duration dd2 taken for decoding the S-ODSslast, which ts the ODS of which 
decoding time Is latest among all S-ODSs needed for presentation of the initial interactive display, 

33 [0130] Further, for presentation of the initial interactive display, the screen must be cleared and decoded graphics 
objects must be transferred. Thus, the PTS (DSn[lCS]) of the ICS needs to be set to the value corresponding to the 
time, at the earliest, calculated by adding to the value of PTS(DSn[S-ODSstast]), the screen clear duration (90,000 x 
(8 x video.wtdth x video.height // 128,000,000) and the decoded object transfer duration (90,000 x (ZSIZE (DSn 
[ICS.BUTTON[fl]) // 128,000,000)). This concludes the description regarding the synchronization control defined by 

40 the ICS. 

[0131] In the case of DVDs, the interactive control ts valid during the time of VOBU, which corresponds to GOP of 
the video stream. In the case of BD-ROMs, however, the valid period of Interactive control can be arbitrarily set with 
the use of the PTS and DTS of the ICS included In an Epoch. That is to say, the Interactive control of BD-pOM Is 
Independent of GOP. 

45 [0132] Note that synchronous control by the PTS of the ICS includes not only displaying a button at a specific point 
on the reproduction time axis, but also making a Popup Menu available for presentation during a specif Ic time period 
on the reproduction time axis . The Popup Menu is a menu displayed at a push of a menu key of the remote controller 
400. The Popup menu becomes available for presentation at the display timing of a specific picture within the AV Clip. 
Such control is also Included in the synchronous control defined by the PTS of the ICS. Similarly to ODSs used for 

so presentation of buttons, ODDS for presentation of the Popup menu are decoded and the decoded objects are rendered 
on the graphics plane. Unless the rendering to the graphics plane Is completed, no response can be made to a menu 
call from users. In order to allow the synchronous display of the Popup menu, the PTS of the ICS shows the time upon 
which the Popup display becomes available. 

[0133] This concludes the description of the recording medium according to the first embodiment of the present 
55 Invention. The following describes a reproduction apparatus according to the first embodiment of the present Indention . 
FIG. 25 shows an Internal structure of a reproduction apparatus. The reproduction apparatus according to the present 
Invention Is industrially manufactured based on this internal structure. The reproduction apparatus Is roughly made up 
of three parts that are a system LSI, a drive device, and a microcomputer system. The reproduction apparatus can be 
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manufactured by mounting these parts on a cabinet and substrate of the apparatus. The system LSI Is an Integrated 
circuit including various processing units for achieving the functions of the reproduction apparatus. The reproduction 
apparatus manufactured in the above manner Is composed of a BO drive 1 , a track buffer 2, a PID filter 3, transport 
buffers 4a, 4b, and 4c, a peripheral circuit 4d, a video decoder 6, a video plane 6, an audio decoder 7, a graphics plane 

5 8, a CLUT unit 9, an adder 10, and a graphics decoder 12, a coded data buffer 13, a peripheral circuit 13a, a stream 
Graphics Processor 14, an object buffer 15, a composition buffer 16, a graphics controller 17, a UO controller 18, a 
player register group 1 9, and a controller 20. . 
[0134] The BO drive 1 performs loading, reading, and ejecting of the BD-ROM so as to access the BD-ROM. 
[01 35] The track buffer 2 is a Fl FO memory. Accordingly, TS packets read from the BD-ROM are removed from the 

10 track buffer 2 in the same order as they arrive. 

[0136] The PID filter 3 performs filtering on TS packets output from the track buffer 2. In more detail, the PID filter 3 
passes only TS packets having predetermined PIDs to the transport buffers 4a, 4b, and 4c. There is no buffering 
required inside the PID filter 3. Accordingly, TS packets entering the PID fitters are written to the transport buffers 4a, 
4b, and 4c without delay. 

15 [0137] The transport buffers 4a, 4b, and 4c are FIFO memories for storing TS packets output from the PID filter 3. 

[0130] The peripheral circuit 4d has a wired logic for converting TS packets read from the transport buffers 4a, 4b, 

and 4c to functional segments. The functional, segments are then stored to the coded data buffer 1 3. 

[0139) The video decoder 5 decodes TS packets output from the PID filter 3 to obtain uncompressed pictures, and 

writes the obtained pictures onto the video plane 6. 
so [0140] The video plane 6 is a plane memory for video data. 

[0141] The audio decoder 7 decodes TS packets output from the PID filter 3, and outputs uncompressed audip data. 

[0142] The graphics plane 8 is a plane memory having a memory area of one screen, and Is capable of storing 

uncompressed graphics of one screen. 

[0143] The CLUT unit 9 converts index colors of the uncompressed graphics on the graphics plane 8, based on Y, 

23 Cr, and Cb values defined in a PDS. 

[01 44] Th e adder 1 0 multiplies the u ncompressed graphics converted by the CLUT unit 9, by a T value (transparency) 
defined in the PDS. The adder 1 0 then performs addition for corresponding pixels in the resulting uncompressed graph-, 
ics and the uncompressed picture data on the video plane 6 to output a composite image. 
[0145] The graphics- decoder 12 decodes a graphics stream to obtain uncompressed graphics, and renders the 

30 uncompressed graphics to the graphics plane 8 as graphics objects. As a result of decoding the graphics stream, 
subtitles and mentis appear on the screen . This graphics decoder 1 2 is composed of the coded data buffer 1 3, the 
peripheral circuit 13a, the stream graphics processor 1 4, the object buffer 1 5, the composition buffer 1 6, and the graph- 
ical controller 17. 

[0146] The coded data buffer 13 is used for storing functional segments together with their DTSs and PTQs. The 

35 functional segments are obtained by removing a TS packet header and a PES packet header from each TS packet 
stored in the transport buffers 4a, 4b, and 4c, and arranging the payloads In sequence. The DTSs and PTSs contained 
in the removed TS packet headers and PES packet headers are stored in correspondence with the PES packets. 
[01 47] The peripheral circuit 1 3a has a wired logic for transferring data from the coded data buffer 1 3 to the stream 
graphics processor 14, as well as from the coded data buffer 13 to the composition buffer 16. In more detail. when the 

40 current time reaches the DTS of an ODS, the peripheral circuit 13a transfers the ODS from the coded data buffer 13 
to the stream graphics processor 14. Also, when the current time reaches the time shown by the DTS of an tC|S or of 
a PDS, the peripheral circuit 1 3a transfers the ICS or PDS from the coded data buffer 1 3 to the composition buffer 16. 
[0148] The stream graphics processor 1 4 decodes an ODS to obtain uncompressed graphics having index colors, 
and transfers the uncompressed graphics to the object buffer 15 as a graphics object. The decoding by the stream 

45 graphics processor 14 begins at the time shown by the DTS associated with the ODS and ends by the decoding . 
deadline shown by the PTS also associated with the ODS. The decode rate Rd of the graphics object decoding men- 
tioned above is equal to the output rate of the stream graphics processor 14. 
[0149] The object buffer 15 stores graphics objects decoded by the stream graphics processor 14. 
[01 50] The composition buffer 1 6 is used for storing an ICS and a PDS. 

so [0151] The graphics controller 1 7 decodes an ICS stored In the composition buffer 16, and carries out control based 
on a decoding result at the timing specified by the PTS attached to the ICS. . 

[01 52} The UO controller 1 8 detects a user operation made on the remote controller and the front panel of reproduc- 
tion apparatus, and outputs Information showing the detected user operation (hereinatter referred to as a UO (User 
Operations))' to the controller 20. 
55 [0153] The player register group 1 9 is a group of registers provided within the controller 20, and includes 32 player 
status registers and 32 general purpose registers. The meanings of player status register values (PRS) are shown 
beJow. The expression "PSR (x) 0 represents the value of the xth player status register. 
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PSR(O) : Reserved 

PSR(1): Audio Stream Number to be decoded 

PSR (2) : Subtitle Graphics Stream Number to be decoded 

PSR{3): Angle Number set by User 

PSR{4) ; Tide Number being Current Reproduction Target 

PSR{5): Chapter Number being Current Reproduction Target 

PSR{6): PL Number being Current Reproduction Target 

PSR{7): Playltem Number being Current Reproduction Target 

PSR(B): Time Information showjng Current Reproduction Point 

PSR(9): Count of Navigation Timer 

PSR(10): Button Number Currently in Selected State 

PSR(11)- (12): Reserved 

PSR(13): Parental Level set by User 

PSR(14): Reproduction Apparatus Setting for Video 

PSR(15); Reproduction Apparatus Setting for Audio 

PSR(16): Language Code for Audio 

PSR(1 7): Language Code for Subtitles 

PSR(18); Language Code for Menu Description 

PSR (19) - (63) : Reserved 
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[0154] The PSR (6) is updated each time a picture belonging to the AV Clip is displayed, That Is, when the reproduction 
apparatus displays a new picture, the PSR(8) is updated so as to hold the value corresponding to the display start time 
(i.e. Presentation Time) of the newly displayed picture. With reference to PRS (8), the current reproduction point is 
known. 

25 [01 55] The controller 20 performs integral control through the bi-directional data exchanges with the graphics decoder 
1 2. The data outputted from the controller 20 to the graphics decoder 1 2 Is UOs received by the UO controller 1 8. The 
data outputted from the graphics decoder 12 to the controller 20 is button commands included in an ICS. 
[0156] The components of reproduction apparatus having the above structure perform processing in a pipeline. 
[01 57] FIG. 26 is a timing chart showing decoding of ODSs. In the figure, the forth level shows DSs recorded on the 

30 BD-ROM . Hie third level shows the read durations of ICSs, PDSs, and ODSs by the coded data buffer 13. The second 
level shows the decode durations of the ODSs by the stream graphics processor 14. Hie first level shows durations 
of processing by the graphics controller 1 7. The decoding start time of each ODS te shown in the figure as DTS11, 
DTS12, and DTS13. Each ODS needs to be read to the coded data buffer 13 by the decoding start time specified by 
a respective DTS. Therefore, the reading of ODS1 Is completed, at the latest, Immediately before the decode duration 

35 dpi of the ODS1 to the coded data buffer 1 3. Similarly, the reading of ODS (n) Is completed, at the latest, immediately 
before the decode duration dp2 of the ODS2. f 
[0158] The decoding deadline of each ODS Is shown In the figure as PTS11, PTS12, and PTS13. The deciding of 
ODS1 by the stream graphics processor 14 is completed by the time shown by the PTS11 , and the decodlng'of ODS 
(n) la completed by the time shown by the PTS12. As above, eaoh ODS is read to the coded data buffer 1 3 by Jhe time 

^o shown by a respective DTS of the ODS, and the read ODS Is decoded and loaded to the object buffer 16 by |ho time 
shown by a respective PTS of the ODS. The above processing Is performed in pipeline by the single stream graphics 



of the Initial Interactive display become available on the object buffer 15 upon completion of the decoding of ajll of the 
45 ODSs for the normal state and the first ODS for the selected state. In the figure, at the time shown by the PTS1 3, all 
of the graphics objects necessary for presentation of the initial interactive display become available. 
[0150] On the first level, the duration cd1 is needed for the graphics controller 1 7 to clear the graphics plane 8, and 
the duration td1 is needed for rendering the graphics on the object buffer 1 6 to the graphics plane 8. The position within 
the graphics plane B for rendering is specified by the button JiorlzontaLposltion and button_vertical_posltlon fieide In 
&> the ICS. That Is to say, uncompressed graphics constituting an Interactive display is obtained on the graphics plane 8 
at the end of time calculated by adding to the PTS13 value of the ODS, the plane clear duration cd1 and the rendering 
duration td1 of the decoded graphics objects. The GLUT unit 9 performs color conversion on the uncompressed graph- 
ics, and the adder 1 0 overlays the graphics on an uncompressed picture stored with the video plane 6. As a result, a 
composite image Is produced. 

& [0161] With the above arrangement, the initial Interactive display Is presented without watting for completion of the 
decoding of the ODSs belonging to the button-state groups for the selected state and the activated state. Thus, In 
comparison with the case where the initial Interactive display is presented only after all the ODSs within the'DS are 
decoded, the presentation is earlier by the duration hy1 shown In the figure. 
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[0162] Note in the figure, there are more than one ODS appended with the numerical subscripts, such as "1" and 
m ri m , to read "ODS 1" and 'ODS n". Yet, each ODS 1 Included In N-ODSs, S-ODSs and A-ODSs is afferent. The same 
holds to the drawings with similar reference numerals. 

[01 63] In the graphics decoder 1 2, the stream graphics processor 1 4 continues decoding while the graphics controller 
5 17 Is clearing or rendering the graphics plane B (the decode durations of ODS (n), ODS 1 , ODS (n) shown on the 
second level) . Since the decoding of the remaining ODSs are continued even during the graphics controller 1 7 is 
clearing or rendering the graphics plane 8, the decoding of the remaining ODSs are completed earlier than would 
otherwise be the case. Thus, updates of the Interactive display using those remaining ODSs can be carried out Imme- 
diately upon a user operation. As a result of such pipeline processing, presentation of the initial Interactive display and 
10 updates to the display can be carried out speedily. 

[0164) The example in FIG. 26 relates to the case where a default selected button is specified. FIG. 27 is a timing . 
chart showing the pipeline processing performed by the reproduction apparatus in the case where no default selected : 
button is specif ted. In such a case, all of the graphics objects needed for presentation the Initial display become available 
upon completion of decoding all the ODSs for the normal state and for the selected state; and loading the resulting 
15 objects to the graphics plane 8. With this arrangement, the initial interactive display is presented without waiting for 
completion of the decoding of the ODSs for the activated state. Thus, in comparison with the case where Interactive 
display is presented only after all of the ODSs within the DS are decoded, the presentation is earlier by the duration 
hy2 shown In the figure. 

[0165] This concludes the disruption of the internal structure of reproduction apparatus. Next, a description is given 
20 to implementation of the controller 20 and the graphics decoder 1 2. The controller 20 can be implemented by a general- 
purpose CPU executing a program of the steps shown by FIGs. 28 and 29. Hereinafter, a description is given 'to op- 
erations performed by the controtter 20 with reference to FIGS. 28 and 29. 

[0166] FIG. 28 is a flowchart of the operations performed by the controller 20 for executing a LinkPL function. Upon 
decoding a command including a LinkPL function, the controller 20 operates as shown in FIG. 20. 

25 [0167] In this flowchart, the Playftem and Access Unit to be processed are denoted as PI (y) and Access Unit (v), 
respectively. As shown In the flowchart, the reproduction apparatus reads the PL (.mpls) specified by the argument of 
the LinkPL (step S1), and designates the first PI in the current PL as a PI (y) (step S2). The reproduction apparatus 
then reads Clip information specified by the Clipjnf6rmation_file_name field in the PI (y) (step S3). 
[0168] Upon reading the Clip Information, the reproduction apparatus converts the value of INJime field within the 

30 pi (y) to an address using the EP_map Included in the Clip Information (step S4). The Access Unit specified by the 
resulting address is designated as an Access Unit (v) (step S5). The reproduction apparatus also converts the value . 
of Outjime field to the address using the EP_map included in the Clip information (step S8). The Access Unit specified 
by the resulting address is designated as an Access Unit (w) (step S7). 

[0169] Once the Access Units (v) and (w) are designated, the reproduction apparatus Instructs the BD drive to read 
35 the Access Units from (v) through (w) (step S8), arid also instructs the video decoder 5, the audio decoder 7, the 
graphics decoder 1 2 to decode the data residing at the addresses corresponding to the ln_time through Outjime fields 
of the PI (y) (step S9) . 

[0170] In a step S11 , an end Judgment of this flowchart is made to see if the PI (y) is equal to PI (z) . If the step S11 
results in "YES*\ the processing of this flowchart Is terminated. If not, the next Play Item is designated as a neiy PI (y) 
*o (step S1 2), and the processing goes back to the step S3. The steps S1 -S10 are repeated until the step S1 1 results in 
"YES". ' 
[0171] In a step S10, a function segment is loaded to the coded data buffer 13 simultaneously with the reading of 
Access Units. 

[01 72] FIG. 29 Is a flowchart of the operations preformed for loading functional segments. In the flowchart, a Segment 
45 (K) is a variable Indicating a segment (ICS, ODS, or PDS) read with an Access Unit, and an Ignore flag indicates 
whether the Segment (K) is to be ignored or loaded: In this flowchart, after the ignore flag Is initialized to 0. (step S20), 
a loop of steps S21 to S24 and S27 to S35 is performed for the Segment (K) (S25 and S26). : 
[0173] In the step S21 , the reproduction apparatus judges whether the Segment (K) is an ICS. If the Segmeijt (K) is 
an ICS, the processing goes to the Judgments in the steps S27 and S28. 
so [0174] In the step $27, a Judgment is made as to whether the segmentjype field within the ICS Is set to a value 
indicating an Acquisition Point DS. If the Segment (K) belongs to an Acquisition Point DS, the processing goes to the 
step S28. On the other hand, If the Segment (K) belongs to either an Epoch Start or Normal Case DS, the processing 
goes to the step S33. 

[0175] In the step S28, a judgment is made as to whether the immediately preceding DS is stored in the coded data 
55 buffer 13. The step S28 is performed If the step S27 results in "YES", The immediately preceding DS is not stored in 
the coded data buffer 13 if a skip operation is performed. In this case, the presentation needs to be started from an 
Acquisition Point DS, so that the processing goes to the step S30 (S28: NO). 

[0176] On the other hand, in the case where the immediately preceding OS is stored in the coded data buffer 13 
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(step S26: YES), the Ignore flag Is set to n 1° (step S29), and the processing goes to the step S31 . 
[0177] In the step S31 , a judgment Is made as to whether the command_update_flag field Is set to "1". If set to "1 0 
(step S31 : YES), the button command of the button Information Is loaded to the coded data buffer 1 3 while the other 
data Is Ignored (step S32) . If the command^updatejlag field Is set to "0", the processing goes to the step S2g. Con- 
5 sequentfy, the ICS indicating an Acquisition Point Is ignored (step S24) . 

[0178] With the Ignore Dag eel to "1", ihe step S22 results in "NO", so that all the functional segments belonging to 
an Acquisition Point DS is Ignored. 

[0179] In a step S33, a judgment is made as to whether the segment.type field within the ICS Indicates a Normal 
Case DS. If the Segment (K) belongs to an Epoch Start OS, the Ignore flag is set to "0" in the step S30. 

w [01 80] If the Ignore flag is set to M 0" (Step S22: YES), the Segment (K) is loaded to the coded data buffer 1 3 (step S23) . 
[0181] On the other hand, If the Segment (K) bolongs to a Normal Case DS, the processing goes to the step S34. 
Similarly to tho step S28, in the step S34, a judgment Is made as to whether the Immediately preceding DS la stored 
in the coded data buffer 13. If the Immediately preceding DS is stored, the Ignore flag is set to "0" (step S30). If not, 
the ignore flag Is set to " 1 " (step S35). With the ignore flag set as above, the functional segments belonging to a Normal 

15 Case DS are ignored if the immediately preceding DS is not stored in the coded data buffer 13 of the reproduction 
apparatus. 

[0182] In the case where the auto_action_fiag field In each piece of button Information (1), (2), and (3) Is set to "1", 
the three buttons go Into the activated stated instead of the selected state, and the button commands (LtnkPL(PL#21)> 
LinkPL(PL#22), and UnkPL(PL#23)) within the pieces of button Information are executed. Suppose, for example, 

so pl#21 , PL#22, and PL#23 to which the button commands are linked are butting and pitching scenes of baseball players, 
those scenes are reproduced upon a numeric Input of a corresponding player number. Since the selection Is made 
with an Input of a player number, which Is easy to recognize for users, the user operabllity Is further enhanced. 
[0183] Now, a description is given to how DSs are read in the example shown in FIG. 30. In this example, three DSs 
(DS1 , DS1 0 , and DS20) are multiplexed with video data. In the DS1 , which is the first of the three DSs, the segmeptjype 

£5 field shows Epoch Start, the command.update Jtag field Is set to "0", and the button command UnkPL(PL#5) is in- 
cluded. 

[0184] The DS 10 is a duplicate of the DS1 . In the DS10, the segmentjype field Is set to a value indicating an 
Acquisition Point DS, the command_update_flag Is set to "0", and the button command UnkPL(PL#5) is included. 
■ [0185] The DS20 Is inherited from the DS1 and Its segmentjype field value indicates an Acquisition Point DS. The 
30 DS20 differs from the DS1 in the button command (LinkPL(PL#10)), so that the command.updatejlag is set fo M 1" to 
signal that the button command Is different. 

[0188] Suppose a skip operation is made to a picture pt1 0 in the AV Clip in which the above three DSs are multiplexed 
with video data. In such a case, the DS1 0 which is closest to the skip destination point Is subjected to the processing 
shown in FIG, 29. Specifically, the judgment In the step S27 results In that the segmentjype Is an Acquisition Point . 

39 DS, but the Immediately preceding DS is not stored In the coded data buffer 13, Accordingly, the Ignore flag Is set to 
0. As a result, the DS10 Is loaded to the coded data buffer 13 as shown in FIG. 31 . Suppose, on the other hanql, a skip 
operation is made to a picture which is located at a later point than the DS10 (arrow hst1 in FIG. 30). In this case, the 
DS20 which lollows the DS10 is loaded to the coded data buffer 13 (arrow hst2 in FIG. 31)). 

[0187] FIG. 33 shows how the DS1 , DS10, and DS20 are loaded when normal reproduction Is performed shown 

40 in FIG. 32. Of the three DSs, the DS1 of which eogmenUype field shows an Epoch Start DS Is loadod to the coded 
data buffer 1 3 (step S23). However, f orthe DS1 0 of which segment type field shows an Acquisition Point DS, the Ignore 
flag is set to T (step S29). Consequently, the functional segments of DS10 are not loaded to the coded date buffer 
13 but ignored (step S24). As for the DS20, the segment_type field shows an Acquisition Point DS, but the 
command.updatejlag Is set to "1*. Thus, the step 31 results in "YES". Consequently, the button command la, loaded 

49 to the coded data Buffer 13, thereby replacing the currently stored button command within the ICS of the DS. Yet, the 
Ignore flag is set to "1 V and thus other data than the button command is not loaded but Ignored. 
[0188] With the above operations, by the time the DS20 Is presented, the button command LlnkPL (#5) associated 
with the same graphics as DS10 has been replaced by LlnkPL (#10). With this replacement, such control la possible 
that the button command associated with the same button changes as the reproduction proceeds. 

so [0189] Next, a description is given to the processing performed by the graphics controller 17. FIG. 34 shows the 
main routine of the processing performed by the graphics controller 17. The flowchart shows that the following three 
operations are repeated: time stamp synchronization (step S35); animation presentation (step S38); and UO processing 
(stepS37). 

[0190] FIG. 35 is a flowchart of the synchronization control carried out based on the time stamps. In the flowchart, 
w judgments are made In the step S43-S47 as to whether specific events have occurred. An occurrence of eaoh event 
calls a respective subroutine which retunes to the main routine after predetermined steps are performed. 
[0191] In the step S43, a judgment is made as to whether the current reproduction point has reached the time shown 
by the PTS of the ODS. If judged in the affirmative, the rendering to the graphics plane 8 Is performed (step S$1), and 
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then the processing goes back to the main routine. 

[01 92] In the step S45, a judgment is made as to whether the current reproduction point has reached the PT3 of the 
ICS. If judged in the affirmative, the graphics plane 8 rs started to output the stored content. The output is madq to the 
CLUT unit 9 where color conversion is made. After the color conversion, the interactive compotatton is overlaid with 

9 the content stored in the video plane 6, thereby an Initial interactive display is presented (step S52). Then, a variable 
q is set to the value "V (step S53), and the processing returns to the main routine. Note that the variable q is a global 
variable (i.e. variable valid throughout a plurality of flowcharts) and its meaning is described later. 
[0193] In the steps S46 and S47. judgments are made as to whether the current reproduction point has reached the 
time specified by the time information within the ICS. 

w [0194] In the step $46, the judgment is made as to whether the current reproduction point has reached the time 
shown by the se!ection_ttme_out_ots. If the judgment results in the affirmative, the button specified by the 
defautt_activated_button_number is activated, and then the processing goes back to the main routine (step S54). 
[0196] In the step S47, the judgment is made as to whether the current reproduction point has reached the time 
specified by the composition jime_out_pts. If the judgment results in the affirmative, the processing retunes to the 

13 main routine after dealing the graphics plane (step S55). This concludes the description of the synchronization based 
on the time stamps. In the synchronization, the steps 51 and 54 call subroutines. The subroutine of the step S51 is 
described below with reference to FIG. 36. 

[0196] FIG. 36 is a flowchart of the rendering to the graphics plane 8. The rendering to the graphics plane 8 needs 
to be completed before the presentation of initial interactive display, in the figure, the ODSx represents the ODS paving 
20 the PTS corresponding to the current reproduction point, which is shown by the PRS (1 0). In steps S61 -563, judgments 
are made as to whether the ODS is the last one of the ODSs necessary for the initial interactive display. If the ODS is 
the last one, the steps S64-S72 are performed. 

[0197] in the step S61 , a judgment is made as to whether the default_selectedJ>utton number field has a valid value. 

If it is valid, a judgment is made in the step S63 as to whether ODSx is S-ODSsfirst. If the step S63 results in "NO", 
25 the processing of this flowchart is terminated and returns to the main routine. 

[0198] If the step S61 results In "NO", a judgment is made in the step S62 as to whether the ODSx is S-ODSslast. 

if the step S62 results in "NO", the processing of this flowchart is terminated, and returns to the main routine. 

[0199] in the step S64. a judgment is made as to whether the segmentjype field within the ICS shows an Epoch 

Start OS. If an Epoch Start DS is shown, the step S65 is performed to clear the graphics plane 8 t and the steps £66-72 
so are performed. The duration taken for clearing the graphics plane 8 is the duration cd1 shown In FIGs. 23 and 24. If 

the segmentjype field does not show an Epoch Start DS, the step S65 is skipped to the steps S66-72. 

[0200] The steps S66-S72 form a loop in which each piece of button information in the ICS is sequentially processed 

(steps S66 and S67). In the loop, the current piece of button Information to be processed Is denoted as buttonjnfo 

(p). In the step S68, a judgment is made as to whether the buttonjnfo (p) corresponds to a default selected button 
as specified by the default_seleded_buttonjiumber. 

[0201] If the buttonjnfo (p) does not corresponding to the default selected button, out of the graphics objects stored 

in the object buffer 16, the one specified in the start_objectjd_normal field of the button Jnfo(p) is designated as the 

graphics object(p)(step S69). 

[0202] On the other hand, if the buttonjnfo (p) corresponds to the default selected button, out of the graphics objects 
40 stored in the object buffer 1 5, the one specified by the 6tart_object Jd.selected field of the buttonjnfo (p) is designated 
as the graphics object (p) (step S70). Then, the button (p) Is designated as the current button (step S71 ). The current 
button refers to the one having the selected state in the currently presented interactive display. The reproduction ap- 
paratus stores the ID of the current button in the PRS(10). 

[0203] Once the graphics object (p) Is designated through the steps S69 and S70, the graphics object (p) is rendered 
43 onto the graphics plane 8 at the position shown by the buttonJtortzontaLposttlon and button jverticalj>osltton fields 
(step S72). The above processing is repeated for each piece of button Information within the ICS. in doing so, of a 
plurality of graphics objects associated with each button state, the first graphics object is rendered for each button on 
the graphics plane 8. The duration taken for rendering all the graphics object stored in the object buffer 1 5 is the juration 
td1 shown in FIGs. 23 and 24. This concludes the description of the step S51. Next, with reference to FIG. 37, a 
so description is given to the subroutine called in the step $54. 

[0204] FIG. 37 is a flowchart of the processing for automatic activation of the default selected button. First of all, a 
judgment is made as to whether the def auh_acttvated_button_number field Is set to "00" or TP (step S91). If the field 
is set to "00", the processing returns to the main routine without performing any operation. On the other hand, if the 
default_activatedj3utton jiumber is set to TP, the current button is designated as the button (u) (step S95). 
35 [0205] If the field value Is neither "00 M nor "FP, the button specified by the default_actlvated_button_number1ield is 
designated as the button (u) (step S92) and activates the button (u) (stepS93) . This button state. transition is carried 
out by rendering the sequence of graphics objects from the one specified by the starLobJect Jd_acttvated flelcj and to 
the one specified by the end_objectJd_activated field in the buttonjnfo (u). The rendering position on the graphic© 
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plane B is shown by the buttonJiorizontaLpositton and button_vert(cal .position fields. Then, the button command 
associated with the button (u) is executed (step S94), and the processing returns to the main routine. 
[0206] Wtth the above operations, the button In the selected state goes into the activated state at the end of a pre- 
determined duration. This concludes the description of the flowchart shown In FIG. 37. 
s [0207] Next, a description Is given to the anlmatedpresentatlon of buttons In the menu (step S36). FIG. 38 Is a 
flowchart of the processing for animated presentation. 

[0208] The initial display has been presented by rendering onto the graphics plane 8, the graphics objects specified 
in the start_objectJd_normal and 6tart w obJect Jd_selected fields of each piece of button Jnfo. To present the buttons 
in animation, for each iteration of the loop of the steps S35-S37, the graphics plane 8 is updated by overwriting each 

w button with an arbitrary frame (the (q)th graphics object) of the button. To be more specific, the update Is made to the 
graphics piano 8 by sequentially rendering the graphics objects specified by tho normal_statejnfo and 
selectod.statejnfo in each piece of button Jnfo. Hero, the variable q is used to refer to an individual graphics object 
specified in the normal_6tateJnfo and eelected.6tate.lnfo fields in each piece of button information. 
[0209] Now, a description is given to the processing for making the animated presentation, with reference to FIG. 

13 38. In the flowchart, the rendering processing is described on the precondition that animation of every button Is pre- 
sented with the same number of frames. This is for the sake of simplicity in the description. The presentation of buttons 
with different numbers of frames requires more complex processing. Further, also for the sake of simplicity in the 
description, the repeat__normalJlag and repeaLselected Jlag fields are both assumed to be set to the value requiring 
the animation to be continuously repeated. 

%> [0210] In a step S80, a judgment Is made as to whether the initial display has been presented. If the Initial display 
has not yet presented, the processing returns to the main routine without performing and operation. If the Initial display 
has been presented, steps S81-S90 are performed. The steps S81-S90 form a loop In which the steps S83-S87 are 
repeated for each piece of buttonjnfo Included in the ICS (steps S81 and S82). 

[0211] In the step S83, a judgment is made as to whether the buttonjnfo (p) corresponds to the current button, 
23 [0212] If the buttonjnfo (p) does not correspond to the current button, the variable q is added to the value of 
starLobject Jdjiormal in theJ>uttonJnfo (p) to obtain an ID(q) (step S84). 

[0213] If the buttonjnfo (p) corresponds to the current button, the variable q is" added to the value of 
6tart_ob]ectJd_selected field in the buttonjnfo (p) to obtain an ID(q) (step S85). 

[021 4] Out of ihe graphics objects stored In the object buffer 1 5, the graphics object (q) Is rendered onto the graphics 
30 plane 8 at the position specified by the button_horizontal_position and the button.verticaLpositlon fields of the 
buttonjnfo (p) (step S87). The above operations are repeated for every piece of buttonjnfo within the ICS (steps S81 
and SB2). ; 
[0216] Through the above loop, among the graphics objects for presentation of the current button in the selected 
state and other buttons In the normal state, the (q) th graphics objects are rendered on the graphics plane 8 for each 
35 button. 

[0216] In the step S88, a judgment is made as to whether the value of the end_ob|ect_ Id.normaf field has reached 
the value obtained by the start.objectjd.normat field + q (step S89). If judged In the affirmative, the variable q Is 
initialized to "0" and the processing returns to the main routine (step S89). if judged in the negative, the variable q Is 
incremented by 1 , and the processing returns to the main routine (step S90). ' 

40 [0217] Through the steps S80-S80, the presentation of each button on the interactive display Is updated using a new 
graphics obj ect each time the loop of steps S35-S37 Is executed. By repeating the steps S35-S37 for multiple times, 
each button Is presented In animation. When presenting the buttons in animation, the graphics controller 17 makes 
the adjustment so as to. present the graphics objects at the rate shown by the animation Jrame_rate_code. Such 
adjustment is made also in the other flowcharts when applicable. This concludes the description of processing for 

45 animated presentation. Next, a description is given to the UO processing shown in the step S37 of the main .routine, 
with reference to FIG. 39. 

[0218] FIG. 39 is a flowchart of the UO processing. In the flowchart, judgments are made In steps S100-S103 as to 
whether specific events have occurred. An occurrence of each event calls a respective subroutine which returns to the 
main routine after predetermined steps are performed. In the step S100, a judgment Is made as to whether the 
so UOjnaskJable field is set to D 1 if the filed Is set to "1", the processing returns to the main routine without performing 
any operation. 

[0219] In the step S101, the Judgment Is made as to whether any of the Move Up, Down, Left, and Right keys Is 
pushed. At a push of any of the keys, another button Is designated as the current button accordingly (step S 104) f 
followed by the judgment as to whether the auto_action Jlag of the newly designated current button is set to "01 " (step 
35 S108). If the auto.action Jlag is not set to "01", the processing returns to the main routine. On the other hand, if the 
auto.actlon Jlag is set to "01 ", the processing goes onto the step S1 06. 

[0220] In the step S1 02, a judgment is made as to whether the Activate key Is pushed. At a push of the Activate key, 
the current button is activated In the step S1 05. This button state transition Is carried out by rendering onto the graphics 
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plane 8 the sequence of graphics objects from the one specffied by the start_objectJd_activated field and the one 
specified by the end_obJect_i deactivated field of the current button. The rendering position in the graphics plane 8 is 
specified by the button JwrtzontaLpoeitton and the button_verticaLposrtion fields of the current button. With this ar- 
rangement, such control is realized that the character serving as a button appears to make motion in response to a 
s user operation. After the button state transition, the button command associated with the current button Is executed 
(step S1 08). 

[6221] In a step S1 03, a judgment ts made as to whether a numeric Input is made. If a numeric input is made (step 
S107), operations are performed according to the Inputted numeral (step S107), and the processing returns to the 
main routine. In FIG. 39, the steps S1 04 and S1 07 each call a subroutine. FtGa.40and41 show the subroutines. Now, 
.10 a description is given to those flowcharts. 

[0222] FIG. 40 is a flowchart of the current button change processing. The flowchart is described on the precondition 
that animation of every button is presented with the 6ame number of frames. This is for the sake of simplicity In the 
description. The presentation of buttons with different numbers of frames requires more complex processing. First, it 
is determined which of upperJ>utton_number, lower_button_number, left_button_number, and rtght_button_number 

15 in the neighborjnfp of the current button corresponds to the pushed key (step S110). 

[0223] Let the button (0 denote the current button and the button (j) denote the button to be designated next as the 
current button (step S1 1 1 ). In a step S1 1 2, a judgment is made as to wheth er the button (J) designated in the step S 1 1 1 
is equal to the button (I). If so, the processing returns to the main routine without performing any operation. If not, steps 
S113-S120 are performed. In the steps S113-S1 20, the button (J) is put into the selected state and the button (i) is put 

20 back to the normal state. Each button state is presented in animation. To this end, a variable r is first initialized to "0" . 
The variable ris used to refer to an individual frame of the animation. lnthestepSt13-S119,the(r)th one of the graphics 
objects associated with the button (i) in the normal state and the (r)th one of the graphics objects associated with the 
button (j) in the selected state is repeatedly rendered onto the graphics plane. 

[0224] Specifically, the ID{r) is calculated by adding the variable r to the value of the start_obJect_id.normal field of 

25 button info (i) (step S114) . After calculating the ID(r) in the above manner, out of the graphics objects stored in the 
object buffer 15, the one having the ID(r) is rendered onto the graphics plane 8 at the position shown in the 
button JiorizontaLpositlon and buttonjverticaLposltion fields of the buttonjnfo(i) (step S115). 
[0225] Next, the ID (r) Is then calculated by adding the variable r to the value of the start_objecUd_selected field of 
the button info (j) (step S116), After obtaining the ID(r) in the above manner, out of the graphics objects storeoj In the 

30 object buffer 15, the one having the ID(r) is rendered onto the graphics plane B at the position shown 'in the 
button JiorizontaLposltlon and button_vertical_posltiori fields of the button_info(|) (step S11 7). 
[0226] In the step S11B, a Judgment is made as to whether the ID obtained by adding the variable r to the value of 
start_obJectJd_normal field is equal to the value of end_obJectJd_hormal field. If the values are not equal, the variable 
r Is incremented (step S120), and the processing goes back to the step S114. The steps S114-S120 are repeated until 

$5 the judgment in the step S11 8 results in "YES". By repeating the above steps, the button state transition is achieved 
such that characters serving as buttons make motion In response to a user operation. When the judgment in the step 
5118 results in "YES", the button (j) is now designated as the current button (step S1 19) and the processing returns 
to the main routine. ■ > 

[0227] FIG- 41 Is a flowchart of the numeric input processing. First, a judgment Is made as to whether th^re is a 

40 piece of button Jnfo (J) having the button_number matching the inputted numeral (step S121). Next, a judgment is 
made as to whether the numericairv_selectable Jlag i field In the button Jnfo (j) is sot to "1 ° (step S1 22). If both the steps 
S 121 and S122 result in "YES", a judgment is made as to whether the auto.actionjlag field in the buttonjnfo (j) is 
set to "or (step S1 23). r 
[0228] If the auto_action_flag field value is not "01 the sequence of graphics objects from the one specifieoj by the 

45 start_obJect_ld_se!ected field to one specified by the end object. id_ selected field associated with, the button (J) is 
sequentially rendered at the position specffied by the buttonjiorizontal_position and button_vertical_posmon fields 
associated with the button (j) (step S124). As a result, the button (j) is put into the selected state. Then, the button (j) 
is designated as the current.button (step S1 25), and the processing returns to the main routine. \ 
[0229] On the other hand, If the auto.actionjlag field value is "01 the current button is activated in the step S1 26. 

so This button state transition is carried out by sequentially rendering the sequence of graphics objects from the one 
specified by the starLobJeclJd^activated field to the one specified by the end_objecUd_actlvated field associated 
with the current button onto the graphics plane 8 at the position specified by the btmon JiorteontaLposition and 
button_vertical_positlon fietds associated with the current button, in a step S127, the button command associated with 
the button (j) is executed, and the processing returns to the main routine. 

55 [0230] if either of the steps S1 21 -S123 results in "NO", the processing returns to the main routine. 

[0231] This concludes the. processing performed by the graphics controller 17 for synchronous display. For presen- 
tation of interactive display, such as popup display, triggered by a user operation, the stream graphics processor 14 
and the graphics controller 1 7 perform the following operations, which are similar to those performed for synchronous 
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display. By doing so, the graphics plane 8 obtains necessary graphics objects. After obtaining the graphics objects, no 
operation Is performed until the current reproduction point reaches the time specified by the PTS attached to the ICS. 
If the UO controller 18 receives a user operation of a menu call after the current reproduction point has paeped the 
time specified by the PTS, the graphics objects stored in the graphics plane 8 are outputted to the GLUT unit 9 where 

9 the outputted graphics objects are overlaid with the video plane. The output performed In synchronization with the UO 
leads lo presentation of the Popup display In response to the menu call. 

[0232] As described above, according to the present embodiment, Epochs each composed of an ICS and ODSs are 
multiplexed in an AV Clip With such an AV Clip, Interactive control Is readily described such that the reproduction 
apparatus performs a specific operation In synchronization with the presentation of a specific frame of the video. In 

10 short, the AV Clip of the present invention Is suitable for describing Interactive control closely bound to the presentation 
timing of video frames. Furthermore, since the Epochs are multiplexed within the AV.CIIp, even when reproduction of 
hundreds of video segments Is requested, It Is not necessary to store all the related Epochs to the rpemory. Epochs 
are sequentially read from the BD-ROM with video packets. Thus, the ICS associated with the video segment currently 
reproduced stays resident in the memory throughout the duration of that video segment. Upon completion of the video 
segment, the Epoch having been reproduced Is removed from the memory and an ICS corresponding to the next video 
segment is then loaded to the memory. Since the Epochs are multiplexed in the AV Clip, the required size of memory 
is kept to a minimum even If the number of Epochs exceeds several hundreds. 

(SECOND EMBODIMENT) 

20 

[0233] A second embodiment of the present Invention relates to the Improvement on a click sound reproduced at 
the time when the button state is changed to the selected state and the activated stated. Suppose, for example, a user 
interact with the characters appearing In the movie serve as buttons as shown in FIGS. 16 and 17. In such a case, If 
the voice of a respective character Is reproduced as a click sound in response to the user operations, the user can 

29 instinctively recognize which of the buttons the user Is operating: With this arrangement, It is easier for users to rec- 
ognize the button operations. Here, a problem arises as to how to simultaneously reproduce the click sound and the 
main sound. The main sound used herein refers to the sound of the movie, such as the speech of the characters and 
background music in the movie. The audio stream carrying the main sound is multiplexed into the AV Clip pith the 
video and graphics streams. The audio decoder 7 decodes the audio stream . At the time of reproducing the click sound, 

30 the main audio needs to be muted. To this end, the operations of audio decoder 7 need to be suspended. However, 
the suspension of audio decoder 7 Involves the risk of producing noise. 

[0234] In order to avoid the production of noise due to the suspension of audio decoder 7, the reproduction apparatus 
has the Internal structure as shown in FIG. 42. The structure shown In this figure Is similar to the one shown In the FIG. 
25 with the addition of a preload memory 21 and a mixer 22. 

35 [0238] The preload memory 21 stores In advance uncompressed LPCM data to be reproduced as s click sound. 
[0236] The mixer 22 mixes the uncompressed LPCM data stored in the preload memory 21 with the outpijt of the 
audio decoder 7. The data is mixed at the rate Instructed by the graphics controller 1 7 Included In the graphics decoder 
12 (see FIG. 25). Since a click sound Is reproduced In accordance with the mixing parameters, it is not necessary to 
suspend the output of audio decoder 7. 

40 [0237] This concludes the description of the reproduction apparatus structure according to the second embodiment. 
[0238] in order to simultaneously reproduce the main sound and the click sound, the decompressed LPQM data 
recorded on the BD-ROM needs to be loaded to the preload memory 21 in advance. Unfortunately, however, the size 
of decompressed LPCM data Is relatively large. For example, less than ten seconds of 16-blt/48 kHz LPCM auplo data 
has the size of one megabyte. 

45 [0239] In order to meet the demands for reducing the size of preload memory 21 , the ICS according to the present 
invention has the data structure as shown in FIG. 43. FIG 43 shows the data structure of ICS for reproducing the click 
sound in the above manner. The data structure In the figure differs from the one shown in FIG. 11 In the buttpnjnfo. 
In each piece of button info, the selected.stateJnfoO and the activated_stateJnfo() each additionally have 
°audlo_specff icationjnf o" field and "audlo_reproduction_control Jnfo* field. 

so [0240] The audlo.spectfteationjnfo field snows a file name or ID of audio data tobe read and reproduced as a dick 
sound by the reproduction apparatus when ihe button associated with the button Jnfo Is put Into a different state. The 
click sound data to be loaded to the preload memory 21 Is specified by the selected_stateJnfpO and 
ectivated_8tate_lnfo () of the button Jnfo. The click sound data loaded to the preload memory is supplied to the mixer 22. 
t0241] The audio j , eproduction_controi Jnfo Is composed of a plurality of mbclngparameters. Each mixing parameter 

S3 shows the rate at which each audio component is to be mixed with the main sound. Each mixing parameters takes on 
a value within a range of 0-1 .0. The click sound data is multiplied by the values shown by the mixing parameter^ before 
being reproduced. In the case where the audio data Is composed of R and L components, the 
eudio_reproductfon_controljnfo field shows separate mixing parameters for R and L components, so that trje mixer 
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22 is instructed to mix the audio data at the mixing rates shown by those parameters. 

[0242] With the provision of the audio_reproduction_controljnfo, the L component of the uncompressed LPCfrf data 
may represent the click sound of button A, while the R component representing the click sound of button B, In this way, 
two different click sounds for two different buttons may be collectively stored as one piece of uncompressed LPCfyl data. . 

* [0243] In addition to the collectively stored dick sounds as.descrtoed above, the button Jnfo (1) Includes the 
audio„reproduction_controUnfo set for output of the R component only, whereas the buttonJnfo(2) Includes the 
audio_reproductlon_controljnfo set for output of the L components only; With this arrangement, simultaneously with 
the state transition of the button A to the selected state, the L component of uncompressed LPCM data is reproduced 
according to the audio reproductlon_controljnfo of the buttonjnfo (1), so that the. click sound of button A is repro- 

10 duced. 

[0244] Similarly, simultaneously with the state transition of the button B to the selected state, the R component of 
uncompressed LPCM data is reproduced according to the aud1o M reproductlon_controlJnfo of the buttonjnfo (2), eo 
that the click sound of button B is reproduced. 

[0243] With reference to FIGs. 44 and 45, a description Is given to a specific example of the dick sound reproduction 
*5 using the above ICS and reproduction apparatus. The example related to the state control information shown in FIGs. 
44A and 44B. the state control information shown in FIG. 44A includes button Jnfo (1 ) and button Jnfo(2). As shown 
by the arrows sy1 and sy2 in the figure, both the buttonjnfo (1) and buttonjnfo 2) specify the same piece of click 
sound data, which Is stereo sound. The audlo_reproduction_controljnf6 of the buttonjnfo (1) includes the mixing 
parameter for L component, whereas the audio.reproductlon^controijnf o of the buttonjnfo (2) Indudes the fnixlng 
20 parameter for R component. 

[0246] FIG. 44B shows the reading process of the ICS including the state control Information. Prior to the ICS, click 
sound data is loaded to the preload memory 21 . 

[0247) FIGs. 45A and 45B illustrate the reproduction control of the click sound data according to the ICS read to the 
preload memory 21 . In the case where the button A associated with buttonjnfo (1 ) is in the selected state, the graphics 

25 decoder 12 controls the audio decoder 7 so thai the dick sound data is reproduced according p the 
audio_reproduction_controljnfo of the button Jnfo (1). As a result, the L component of the stereo click sound o'ata is 
reproduced. On the other hand, In the case where the button B associated with button Jnfo(2) is in the selected state, 
the graphics decoder 12 controls the audio decoder 7 so that the click sound data is reproduced according to the 
audio_reproduction_control info of the buttonjnfo (2). As a result, the R component of the stereo click sound data 

so is reproduced. 

[0248] With the above control, a piece of click sound data composed in stereo is used as the click sound of button 
A by reproducing the L component and as the click sound of button B by reproducing the R component, when a re- 
spective button is selected. 

[0249] This concludes the specific example tn which a plurality of dick sounds Is collectively stored as one piece of 

33 uncompressed LPCM data. Alternatively, the buttonjnfo of the present invention may be set so as to produce the dick 
sound upon a button operation In a manner that the sound comes from a different direction. A specific example pf such 
control is described below with reference to FIGs. 45. FIG. 45C shows an example of three pieces of buttonjnfo 
associated with three lateralyalfgnedbuttons (button A, button B, and button C). The buttonjnfo associated vyith the 
button A, which is on the left, has the mixing parameter of 1 ,0 for the L component The buttonjnfo associated with 

to the button B, which is In the middle, has the mixing parameters of .0.5 for both the L and R components. The 1 button 
info associated with the button C, which Is on the right, has the mixing parameter of 1 .0 for the R component. Vyith the 
above mixing parameters, when the left button A Is selected, the click sound is outputted from the left speaker. When 
the right button C is selected, the dick sound Is outputted from the right speaker. When the middle button B is selected, 
the click sound is outputted equally from both speakers. As above, the buttonjnfo may be set so that the dicl$ sound 

45 is outputted from a different direction depending on the position of button on the screen. The dick sound coming from 
the direction corresponding to the position of the button pushed adds to the realism (n the button operations. 
[0250] As described above, according to the present embodiment, a dick sound for each of a plurality of buttons is 
integrated into one piece of stereo click sound data. With the use of audio_spectfication_inft> and 
audio_reproducrJon_control Jnfo, the same piece of click sound data is reproduced as different click sounds for different 
• so buttons. With this integration, the amount of click sound data is reduced, and thus the size of preload memory 21 for 
loading the click sound data can be reduced. ■ 
[0251] Although the specific examples given above are based on the stereo click sound data, the click sound data 
may be uncompressed 5.2-channel audio data. FIG. 44C shows an example similar to the one shown in FIG. 45C. 
Yet, in the example in FIG. 44C, the mixing parameters are set for reproduction of 5.2-channel audio data. Thus,. In 

w addition to the L and R components, the 5. 2-channel audio data includes Center, Rear Left, and Rear Right compo- 
nents. The buttons A, B, and C are diagonally aligned on the interactive display. In this example, the buttonjnfo as- 
sociated with the button A has the mixing parameter of 1 .0 for L component. The buttonjnfo associated with the button 
C has the mixing parameter of 1 .0 for Rear Right component. The buttonjnfo assodated with the button B has the 
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mixing parameters of 0.1 , 0. 1., 0.4, 0.2, and 0.2 for L, R, Center, Rear Left, Rear Right components, respectively. With 
this setting, when the button A is selected, the click sound Is outputted from the right* When the button B is selected, 
the click sound Is outputted from the (eft When the middle button B Is selected, the click sound Is outputted from all 
the directions. As above, the direction from which the click sound comes is changed depending on the poelttop of the 
3 button pushed. This arrangement adds to the realism in button operations (Note In the above example, the buttjpn Jnfo 
associated with the button B may alternatively be set to have the parameter of 1.0 for the Center component find the 
parameter of 1 .0 for each of the other components). 

[02S2] Furthermore, in addition to the audio decoder 7, another audio decoder may be provided for click sound data. 
In this case, the preload memory 21 stores compressed audio data in advance. In response to the button state transition, 
io the audio decoder for click sound data extracts the compressed audio data from the preload memory 21 and decodes 
the extracted data. The provision of the audio decoder for click' sound data allows the preload memory 21 to store the 
compressed audio data, so that the sizo of preload memory 21 can bo reduced. 

{THIRD EMBODIMENT). 

13 

[0253] A third embodiment of the present Invention relates to a manufacturing process of the BD-ROM. FIG. 46 is a 
flowchart of the manufacturing process of the BD-ROM according to the present Invention. 
[0254] The manufacturing process includes a material production step of recording video, sound, and the like (step 
S201 ), an authoring step of creating an application format using an authoring device (step S202), and a pressing step 
20 of creating an original master of the BD-ROM and performing stamping and bonding to complete the BD-ROM (step 
S203). 

[0255] The BD-ROM authoring step Includes the following steps S204-S209. 

[0256] First, in the step S240, videomaterlals, audiomaterlals, and supplemental graphics materials are encoded 
Into a video stream, an audio stream, and a graphics stream, respectively. Next, In the step S205, the verification test 
25 of the graphics stream is carried out. As described in the first embodiment, the graphics stream Includes, in addition 
to graphics data for presentation of buttons, state control Information of the button. This makes It possible to verify the 
graphics stream Independently, if any error Is detected (step S206: NO), an appropriate correction is made to the 
graphics stream, which is not yet multiplexed with the other streams (step S207), and the verif ication test of the graphics 
streams is carried out again. 

30 [0257] If no error Is detected through the verification test of graphics stream (step S206 : YES), the video, audio, 
graphics streams, which are obtained in the step S208 by encoding the materials, are interleaved multiplex converted 
to one digital stream, in the step S209 that follows, necessary information is generated based on the scenario for the 
BD-ROM, and the scenario and digital stream are adapted to be In compliance with the BD-ROM format. I 
[0256] According to the present embodiment, the ICS defining the button state transition is integrated Into the graphics 

33 stream with graphics data, Thus, it Is no longer necessary to wait for the video stream to be encoded or the multiplexing 
of streams to be completed. Instead, the graphic stream Is ready for verification test upon its production to see how 
the button states change as the reproduction proceeds. Since the verification test of button state transition can be 
carried out at an earlier stage of authoring, a possibifity Is avoided that an error Is detected right before the shipment, 
which forces the developers to rush. The verification test of graphic stream alone provides a better environment for 

40 incorporating animated buttons of complex motion into a movie. 

(SUPPLEMENTAL NOTE) 

[0250] Practicing of the present invention is not limited to the specific examples described above. The present In- 
45 vention can be practiced with any of modifications (A) to (O) below. The Invention of each of the claims of this application 
broadened or generalized descriptions of the above-described embodiments and their modifications Including the fol- 
lowing. The extent of the broadening and generalization reflects the state of the art in the related technical field at the 
time of filing of the present application. Yet, the invention recited in each claim is directed to means of solving the 
problems associated with the present invention. Thus, the scope of each Invention does not go beyond the qcope In 
so which those skilled In the art recognize the means lor solving the problems. Consequently, the Invention recited in each 
of the appended claims is in substantial correspondence with the detailed description above. 

(A) The above embodiments describe the case where the BD-ROM Is used as the recording medium. Main features 
of the present Invention, however, lie In a graphics stream recorded on the recording medium, which does not rely 
55 on physical characteristics of BD-ROMs. Therefore, the present invention is applicable to any recording medium 
that Is capable of recording a dynamic scenario and a graphics stream. Examples of such a recording medium 
Include: an optical disc such as a DVD-ROM, a DVD-RAM, a DVD-RW, a DVD-R, a DVD+RW, a DVD+R, ? CD-R, 
or a CD-RW; a magneto-optical disk such as a PD or an M0; a semlcond uctor memory card such as a CompactFlash 
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card, a SmartMedia card, a Memory Stick card, aMultlMediaCard, or a PCMCIA card; a magnetic disk suqh as a 
flexible disk, SuperDlsk, Zip, or CliW; a removable hard disk drive such as ORB, Jaz, SparQ, SyJet, EZFley, or 
Microdrive; and a non-removable hard disk drive. 

(B) The above embodiments describe the cases where the reproduction apparatus decodes an AV Clip on the 
BD-ROM and outputs the decoded AV Clip to the television. As an alternative, the reproduction apparatus may be 
equipped with only a BD drive, with the remaining construction elements being provided in the television. In this 
case, the reproduction apparatus and the television can be incorporated in a home network connected with an 
IEEE 1394 connector. Furthermore, the above embodiments describe the cases where the reproduction apparatus 
is connected to the television, but the reproduction apparatus may instead be integrated with a display device. 
Also, the reproduction apparatus may include only the elements relating to the essential part of processing. Those 
reproduction apparatuses are all descried in this specification as inventions. Accordingly, regardless of the modes 
therefor, an act ot manufacturing a reproduction apparatus based on the internal structure of the reproduction 
apparatus described in the first, second, or third embodiment is considered to be an act of practicing the present 
Invention. Also, any act of assigning with charge (i.e. for sale) or without charge (I.e. as a gift), leasing, and importing 
the reproduction apparatus is an act of practicing the present invention. Likewise, an act of offering for assignment 
or lease of the reproduction apparatus using storefront displays, catalogs, or brochures is an act of practicing the 
present invention. 

(C) Information processing using the programs shown in the flowcharts is actually realized using hardware re- 
sources. Accordingly, the programs which describe the operational procedures shown in the flowcharts are them- 
selves an Invention. The above embodiments describe the cases where the programs are Incorporated In the 
reproduction apparatus, but the programs can be used independently of the reproduction apparatus. Acts of prac- 
ticing the programs Include (1) an act of manufacturing, (2) an act of assigning with or without charge, (3), an act 
of leasing, (4) an act of importing, (5) an act of providing to the public via a bi-directional electronic communications 
network, and (6) an act of offering for assignment or tease using storefront displays, catalogs, or brochures: 

(D) The time elements of the steps which are executed in a time series in each of the flowcharts can be regarded 
as the necessary elements of the present Invention. With this being so, the procedures shown by these flowcharts 
are considered to disclose reproduction methods. If the processing shown In each flowchart is carried out by 
performing the steps in a time series so as to achieve the intended aim and the intended effect, this is an act of 
practicing the recording method of the present invention. 

(E) When recording an AV Clip on the BD-ROM, an extension header may be added to each TS packet tn the AV 
Clip. The extension header is called a TP_extra_header, includes an arrtval_tlme_starnp and a 
copy_perml5Sion_Jndicator, and has a data length of 4 bytes. TS packets with TP_extra_headers (hereafter "EX 
TS packets") are grouped In units of 32 packets, and each group is written to three sectors. One group made up 
of 32 EX T8 packets has 6,144 bytes (=32 x 192), which is equivalent to a size of three sectors that is 6, 144 
. bytes (=2,048 x 3). The 32 EX TS packets contained in the three sectors are called an Aligned Unit. 

In a home network connected with an IEEE 1394 connector, the reproduction apparatus 200 transmits an 
Aligned Unit in the following manner. The reproduction apparatus removes a TP_extra_header from each of the 
32 EX TS packets In the Aligned Unit, encrypts the body of each TS packet according to the DTCP Specif (cation, 
and outputs the encrypted TS packets. When outputnng the TS packets, the reproduction apparatus insprts an 
isochronous packet between adjacent TS packets. A point where the Isochronous packet is inserted is determined 
based on a time shown by an arrrvaLtime_stamp of the TP_extra_header. The reproduction apparatus 200 outputs 
a DTCP_descriptor, as well as the TS packets . The DTCP.descrtptor corresponds to a copy_permlssion_irjdicator 
in the TPlextra_header. With the provision of the DTCP_descriptor indicating "copy prohibited 0 , ft is possible to 
. prevent, when using the TS packets in the home network connected with the IEEE 1 394 connector, the TS packets 
from being recorded to other devices. 

(F) The above embodiments describe the cases where an AV Clip of the BD-ROM Format Is used as a digital 
stream, but the present invention can also be realized with a VOB (Video Object) of the DVD-Video Format or the 
DVD-Video Recording Format The VOB is a program stream that complies with the ISO/IEC 13818-1 Standard 
and is obtained by multiplexing a video stream and an audio stream. Also, the video stream in the AV Clip jmay be 
an MPEG4 video stream or a WMV video stream. Further, the audio stream in the AV Clip may be a Linear PCM 
audio stream, a Dolby AC-3 audio stream, an MP3 audio stream, or an MPEG-AAC audio stream. 

(G) The video editing described tn the above embodiments may be obtained by encoding an analog image signal 
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broadcast by analog broadcasting. 

Alternatively, analog/digital image signals recorded on a videotape may be encoded to obtain contents Also, 
analog/digital Image signals directly captured by a video camera may be encoded to obtain contents. A digital work 
distributed by a distribution server is applicable too. 

5 

(H) The ICS may define the operations to be performed at the time of timeout. The timeout of the ICS Is specified 
by the composttton.tlme.ouLpts described in the first embodiment. FIG. 47 shows an ICS according to one mod- 
ification of the present invention. The ICS in the figure Is newly provided with a Still/Pause Jnformation field. The 
Stlll/Pausejnformatlon field shows whether the operations of reproduction apparatus are to be brought info "Still" 

io or "Pause". The operations of the reproduction apparatus referred herein include the decoding operations by the 

video decoder 5, stream graphics processor 14, and graphics decoder 12 as well as the navigation operations by 
the graphics controller 1 7 and the controller 20. The term "Still" used herein refers to suspension of both the ftecode 
operations and the navigation operations, whereas the term "Pause" refers to suspension of the decode operations 
while the navigation operations are continued. Under the Still condition, the navigation operations are suspended, 

15 so that the last reproduced picture remains displayed as a still picture and no button state transition (s possible. 

. On the other hand, under the Pause condition, the navigation operations are continued, so that users are 
allowed to change the button states. With provision of the Pause/Stlltjnformatlon in the ICS, control to be executed 
upon timeout can be defined at the time of authoring. 

20 (|) tn the second embodiment, the button Jnfo defines a click sound for each button. In addition, a click sound may 
be defined by ICS for each key of the remote controller. FIG. 48 Is a view showing the ICS defining a click sound 
for each key of the remote controller. 

The upper.audio field defines the audio specification Information and reproduction control Information. to be 
referred to at a push of the Move Up key. ' 
25 The lower_aud(o field defines the audio specification information and reproduction control information to be 

referred to at a push of the Move Down key. 

The left.audio field defines the audio specification information and reproduction control Information to be re- 
f erred to at a push of the Move Left key. 

The right_audlo field defines the audio specification information and reproduction control information to be 
30 referred to at a push of the Move Right key, 

The actlvated_audlo field defines the audio specification information and reproduction control to be referred 
to at a push of the Activated key. At a push of a key of the remote controller 400, the preload memory 21 and the 
mixer 22 operate according to the audio specification information and reproduction control Information fhat are 
associated with the pushed key, so that a corresponding click sound Is reproduced. 

35 

(J) Graphics objects described in the above embodiments is run-length encoded raster data. Run-length encoding 
is used for compression/encoding of graphics objects, because the run-length encoding is suitable for compression 
and decompression of subtitles . Subtitles have a property In that a continuous length of the same pixel value In 
a horizontal direction Is relatively long. Therefore, by performing compression using run-length encoding, a high 
40 compression rate can be attained. In addition, run-length encoding reduces a load for decompression! and Is 
therefore suitable for realizing decoding by software. Nevertheless, the use of run-length enooding for graphics 
objects is not essential to the present invention. For example, graphics objects may bePNGdata, Also, graphics 
objects may be vector data Instead of raster data. Further, graphics objects may be transparent patterns] 

45 INDUSTRIAL APPLICABILITY 

[0260] A recording medium and a reproduction apparatus according to the present Invention realize interactive control 
on a movie, thereby adding value to the movie. Movies distributed with such added value Invigorate the movie market 
and consumer appliance market. Thus, the recording medium andthe reproduction apparatus according to the present 
so Invention are highly applicable in the movie and the consumer appliance industries. 



Claims 

55 1, A recording medium comprising: 

a digital stream Into which a video stream and a graphics stream are multiplexed; 
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wherein said graphics stream Includes graphics data and state control information Integrated therein, the 
graphics data being used to compose an interactive display, and the state control Information being used to cause 
• the interactive display to change to a different state in response to a reproduction proceeding of the video stream 
and a user operation. 

5 

2. The recording medium according to Claim 1 , wherein: 
said graphics stream is a packet string; . 

the packet string Includes at least one set of packets, each set of packets includes a packet containing state 
io control information and a packet containing graphics data; and • 

the packet containing state control information is attached with a time stamp indicating a display timing of a 
picture to be displayed in synchronization with the interactive display. 



3. The recording medium according to Claim 1 / wherein: 



15 



said graphics stream includes a plurality of display sets, each of the plurality of display sets Including state 

control information and graphics data; 

the state control Information includes type information; and 

the type Information is used to indicate whether a display set to which the state control information belongs is 
20 identical in contents to an immediately preceding display set on said graphics stream. 

4. The recording medium according to Claim 3, wherein: 

the state control Information in each of the plurality of dfsplay sets includes an update flag; 
25 the update flag, when set to ON, Indicates that the display set is identical to an immediately preceding display 

set with respect to the state control information and the graphics data, except for a button command; 

the update flag, when set to O FF, indicates that the display set is identical to an immediately preceding display 

set with respect to the state control information and the graphics data; and . 

the button command is for execution by a reproduction apparatus upon activation of an associated bufton on 
30 the Interactive display. 

5. The recording medium according to Claim 1 , wherefn: 

the interactive display presents n buttons, and an arbitrary button / of the n buttons has m states, wherein /, 
& n, and m are integers; 

the state control information Includes n pieces of button information, and a piece of button information asso- 
ciated with the button / includes m pieces of state information; and. 

when an arbitrary one of the m states of the button / is a state /, ayth piece of state information indicates which 
piece of the graphics data is to be used for presentation of the button / in the state /, wherein / is an Integer. 

40 

6. The recording medium according to Claim 5, wherein: 

the button information / Includes neighbor Information; and 
. the neighbor information Indicates one of the n buttons to receive a currently selected state in response to a 
45 user operation specifying a direction when the button / is in the currently selected state. 

7. . The recording medium according to Claim 5, wherein: 

each of the n buttons has a number assigned thereto; and 
so each of the n pieces of button information includes a number assigned to an associated button and a flag 

showing whether the button is numerically selectable with the number. 

8. The recording medium according to Claim 5, wherein: 

& the button information 7 includes click sound control information; and 

the click sound control Information indicates a piece of audio data to be reproduced upon a state transition of 
the button /, and indicates a control to be executed by a reproduction apparatus for the audio data reproduction. 
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9. The recording medium according to Claim 1 , wherein: 
said graphics stream Is a packet string; 

the packet string includes at least one set of packets, each set of packets Includes a packet containing state 
5 control information and a packet containing graphics data; 

the packet containing slate control Information Is attached with a time stamp; and 

the time stamp indicates, on a reproduction time axis of the video stream, a time after which the Interactive 
display is available for presentation in response to a user operation. 

10 10. A reproduction apparatus for reproducing a digital stream Into which a video stream and a graphics stream are 
multiplexed, said reproduction apparatus comprising: 

a video decoder operable to decode the video stream to obtain video data; and 

a graphics decoder operable to decode the graphics stream to obtain an Interactive display, wherein: 

is 

the graphics stream includes graphics data and state control information; 

the state control information being used to cause the Interactive display to change to a different state In 
response to a reproduction proceeding of the video data and a user operation; and 
said graphics decoder including 

20 

a processing unit operable to decode the graphics data Included in the graphic stream to obtain the 
interactive display, and 

a controller operable to control a state of the Interactive display based on the state control Information . 
25 n. The reproduction apparatus according to Claim 10, wherein: 
the graphics stream is a packet string; 

the packet string includes at least one set of packets, each set of packets includes a packet containing state 
control Information and a packet containing graphics data; 
so the packet containing state control information is attached with a time stamp indicating a display timing of a 

picture to be displayed In synchronization with the interactive display; and 

the decoding by said processing unit and the control by said controller are performed with reference to the 
time stamp attached to the packet containing state control Information. 

35 12. The reproduction apparatus according to Claim 10, wherein: 

the graphics stream includes a plurality of display sets, each of the plurality of display sets Including q packet 
containing graphics data and a packet containing control information; ' 
the state control information includes type Information indicating a type of the display set; 
40 when performing normal reproduction, said controller ignores a display set of which type Information indicates 

that the display set is identical In contents to an Immediately preceding display set; and . 
when performing reproduction starting from a point searched in a skip operation, said controller decodes a 
display set that is located after the reproduction start point and that has type Information Indicating that the 
display set is identical in contents to an immediately preceding display set. 

45 

13. The reproduction apparatus according to Claim 12, wherein: 

the state control information in each of the plurality of display sets includes an update flag; and 
when performing normal reproduction, said controller reads a button command If the update flag is set to ON 
so and the type information indicates that the display set Is identical In contents to an Immediately preceding 

display set. ' 

14. The reproduction apparatus according to Claim 10, wherein: 

» the Interactive display presents n buttons and an arbitrary button /of the n buttons has m states, wherein I, n, 

end m are Integers; 

the state control Information Includes n pieces of button information, and a piece of button Information asso- 
ciated with the button / includes m pieces of state Information', and 
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when ah arbitrary one of the m states of the button / is a state /, said controller presents the button / In the 

- state I using a piece of graphics data specified by a fih piece out of the m pieces of state information, wherein 
/ is an integer. 

f ' • - * ■ 

9 15. The reproduction apparatus according to Claim 1 4, wherein: 

\ the button Information / includes neighbor information; and 

; when the button / is in a currently selected state and a user operation is made to transfer the currently selected 

state, said controller puts the button / back Into a normal state and puts a button specified by the neighbor 
to information out of the n buttons into the currently selected state, 

16. The reproduction apparatus according to Claim 10, wherein when a user operation of numerical input is made, 
said controller changes a state of a button to which a value equal to the numerical input is assigned. 

ts 17. The reproduction apparatus according to Claim 1 6, further comprising; 

an audio decoder operable to decode an elementary stream multiplexed in the digital stream for producing an 
audio output; 

a preload memory operable to store uncompressed audio data; and 
20 a mixer operable to, upon a user operation to activate a button, mix the uncompressed audio data stored in 

said preload memory with an output of an audio encoder for producing an audio output. 

18. The reproduction apparatus according to Claim 1 7, wherein: 

25 the state control information Includes a plurality of pieces of button information; 

each of the plurality of pieces of button information includes click sound control information; 
the click sound control information indicates a piece of audio data to be reproduced upon a button state tran- 
sition, and indicates a control to be executed by said reproduction apparatus for the audio data reproduction; 
and 

. 30 when a user operation is made to activate a Ath button on the interactive display, the mixer performs the mixing 

■ in accordance with click sound control Information Included In a Mh piece of button information, wherein k is 

an Integer ♦ 

19. The reproduction apparatus according to Claim 1 0, wherein; 

'.35 

the graphics stream is a packet string; 

the packet string includes at least one set of packets, each set of packets includes a packet containing state 
control information and a packet containing graphics data; 
. the packet containing the state control information is attached with a time stamp; 
40 the time stamp indicates/ on a reproduction time axis of the video stream, a time after which the interactive 

; display is available for presentation in response to a user operation; 

said reproduction apparatus further comprises a receiving unit operable to receive a user operation; and 
said controller executes control to present the interactive display when said receiving unit receives a user 
operation after a current reproduction point reaches the time shown by the time stamp attached to the packet 
45 containing state control information, 

' 20. A method for recording data onto a recording medium, said method comprising: 

generating application data; and 
so recording the application data onto the recording medium, wherein: 

i 

- > ' the application data includes a digital stream Into which a video stream and a graphics stream are multi- 
, plexed; and 

the graphics stream Includes graphics data and state control information Integrated therein, the graphics 
55 data being used to compose an interactive display and the state control information being used to cause 

the interactive display to change to a different state in response to a reproduction proceeding of the video 
stream and a user operation. 
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21 . A program for enabling a computer to reproduce a digital stream into which a video stream and graphics stream 
are multiplexed, said program comprising code operable to cause the computer to perform: 

decoding of the video stream to obtain video data; and 
5 decoding of the graphics stream to obtain an Interactive display, wherein: 

the graphics stream includes graphics data and state control Information; 

the state control information is used to cause the Interactive display to change to a different state In re- 
sponse to a reproduction proceeding of the video data and a user operation; and 
10 said decoding of the graphics stream Includes 

decoding the graphics data included in the graphics stream to obtain the Interactive display, and 
controlling a state of the Interactive display in accordance with the state control information. 

15 22. A method for reproducing a digital stream into which a video stream and a graphics stream are multiplexed, said 
method comprising: 

decoding the video stream to obtain video data; and 

decoding the graphics stream to obtain an Interactive display, wherein: 

20 

the graphics stream includes graphics data and state control Information; 

the state control information is used to cause the interactive display to change to a different state In re- 
sponse to a reproduction proceeding of the video data and a user operation; and 
said decoding the graphics data includes 

25 

decoding the graphics data included In the graphics stream to obtain the Interactive display, and 
controlling a state of the interactive display In accordance with the state control information. 
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